Function index (MATLAB)

`/src`

`rs_addtree_choicedata(data_comp, aux)`

[ad,aux_out]=rs_addtree_choicedata(data_comp,aux) analyzes a set of triadic choices to determine the addtree index (a measures of consistency with the four-point inequality, a necessary condition for distances to be consistent with an additive tree). Thi is described in Ordinal Characterization of Similarity Judgments on arXiv and Mathematical Neuroscience and Applications

The analysis is carried out for a range of criteria for the triads to include, and for Dirichlet fits to the choice probability distribution based on all triads (in 'ad.global'), or only the triads that meet threshold criteria (in 'ad.private'). See note below regarding threshold criteria and global vs. private analyses.

Input arguments:

data_comp (int 2-D array) –
Triadic choice data, with each row containing the data from a single kind of comparison
- col 1 is reference, col 2 is s1, col 3 is s2
- col 4: number of times the first difference was judged more similar than the second difference
- col 5: number of times the comparison was made
aux (struct) –
a structure, can be omitted, with fields
- opts_addtree (struct) – options for analysis, with fields
  - if_log (int) – 1 to log progress, 0 to omit; default is 1
  - h_fixlist (float 1-D array) – values for discrete component 'h', should include zero and be in ascending order, default is [0 0.001 0.01 0.1]
  - ntriplets_min (int) – minimum number of triplets for an analysis, default is 3
  - ntents_min (int) – minimum number of tents for an analysis, default is 3
  - if_private (int) – 1 to also do calculations with Dirichlet fits only to the triads that meet threshold criteria, 0 to omit; default is 0
- opts_check (struct) – options for consistency checking, with field
  - if_warn (int) – 1 to show warnings, 0 to suppress; default is 1

Output arguments:

ad ( struct ) –
analysis results, a structure with fields
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_addtree (struct) – aux,opts_addtree with defaults and values used
- opts_check (struct) – aux.opts_check, with defaults filled in
- opts_dirfit_a (struct) – options used for rs_dirfit_choicedata for fitting Dirichlet parameter 'a'
- opts_dirfit_ah (struct) – options used for rs_dirfit_choicedata for fitting Dirichlet parameters 'a' and 'h'
- opts_triplike (struct) – options used for psg_umi_triplike
- fig_handles (cell 1-D array) – handles to figures for summary plot ((present only if if_plot>=1)
- fig_handle_detailed (handle) – handle to figure for detailed plot (present only if if_plot=2)

Note regarding thresholds and global vs. private analyses

Triplets are screened by a threshold criterion based on the number of trials before inclusion in the calculation of the symmetry and ultrametric indices.
The criterion is applied three ways: to the minimum numnber of trials of the three triads in a triplet, the maximum number, and the average number
For the 'global' analysis, the selected triplets are used to calculate the symmetry and ultrametric indices, but all triads are used to calculate the Dirichlet parameters
For the 'private' analysis, the selected triplets are used to calculate the symmetry and ultrametric indices and also to calculate the Dirichlet parameters
The 'private' analysis is substantially slower than the 'global' analysis, and values are of the indices are typically similar; it is only enabled by setting opts_addtree.if_private=1

Triads, trials, triplets, and tents

A triad is a set of three stimuli used in a triadic judgment: one stimulus is the reference, and the other two are the comparison stimuli
A trial is a single judgment of similarity for a given triad
A triplet is a set of three triads built out of the same three stimuli, in which each stimulus in turn serves as the reference
The number of trials in a triplet is the sum of the number of trials in its three triads
A tent is a set of six triads built out of four stimuli. Three of the triads are a triplet built from the first three stimuil; the other three triads are triads built from the fourth stimulus and two of the first three.

See also: rs_dirfit_choicedata, rs_symumi_choicedata, PSG_TRIPLET_CHOICES, loglik_beta_discrete.

Source code in src/rs_addtree_choicedata.m

function [ad,aux_out]=rs_addtree_choicedata(data_comp,aux)
% [ad,aux_out]=rs_addtree_choicedata(data_comp,aux) analyzes a set of triadic choices to determine the addtree index
% (a measures of consistency with the four-point inequality, a necessary condition for distances to be consistent with an additive tree).
% Thi is described in Ordinal Characterization of Similarity Judgments on [arXiv](https://arxiv.org/abs/2310.07543)
% and [Mathematical Neuroscience and Applications](https://mna.episciences.org/16310/pdf)
%
% The analysis is carried out for a range of criteria for the triads to include, and for Dirichlet fits to the choice probability distribution
% based on all triads (in 'ad.global'), or only the triads that meet threshold criteria (in 'ad.private').  See note below regarding threshold criteria and global vs. private analyses.
%
% Args:
%   data_comp (int 2-D array): Triadic choice data, with each row containing the data from a single kind of comparison
%
%      - col 1 is reference, col 2 is s1, col 3 is s2
%      - col 4: number of times the first difference was judged __more similar than__ the second difference
%      - col 5: number of times the comparison was made
%
%   aux (struct): a structure, can be omitted, with fields 
%
%     - opts_addtree (struct): options for analysis, with fields
%
%         - if_log (int): 1 to log progress, 0 to omit; default is 1
%         - h_fixlist (float 1-D array): values for discrete component 'h', should include zero and be in ascending order, default is [0 0.001 0.01 0.1]
%         - ntriplets_min (int): minimum number of triplets for an analysis, default is 3
%         - ntents_min (int): minimum number of tents for an analysis, default is 3
%         - if_private (int): 1 to also do calculations with Dirichlet fits only to the triads that meet threshold criteria, 0 to omit; default is 0
%
%         - **Plotting and replotting**
%         - if_plot (int): 1 for basic plot and statistical summary, 2 for detailed plot, 0 to omit; default is 1
%         - plot_label (char): string for plot label, default is 'ordinal analysis'
%         - ad (struct): include to replot a previous analysis; otherwise omit
% 
%         - **Options for statistics and shuffles**
%         - if_frozen (int): random number control; 1 for same numbers every run, 0 for different random numbers each run, negative integer for a fixed seed each run, default is 1
%
%         - **Options to control optimization details**
%         - a_limits (float): allowed range for 'a' (shape), when fitting only 'a', default is [10^-2 10^2]
%         - a_optimset (struct): non-default optimizations parameters for fitting 'a', with `fminbnd`, default is struct()
%         - ah_optimset (struct): non-default optimizations parameters for fitting 'a' and 'h', with `fminsearch`, default is struct()
%
%         - **Options for internal use and maintenance**
%         - if_fast (int): use hard-coding for partitions, skip calls to filldefault,  default is 1
%         - if_check (int): 1 to compare methods, -1 to log, treated as 0 if if_fast=1, default is 0
%         - tol (float): tolerance for checking consistency, ignored if if_check is 0, default is 10^-5
%         - if_vec (int): if present, use vectorized method, default is abesent
%         - if_partition (struct): if present, use general calculation of psg_ineq_logic, default is absent
%
%     - opts_check (struct): options for consistency checking, with field
%
%         - if_warn (int): 1 to show warnings, 0 to suppress; default is 1
%
% Returns:
%   ad (struct): analysis results, a structure with fields
%
%     - **Addtree indices**
%     - summary (cell 1-D array): summary of addtree index, organized of by fraction of triplets retained; summary{1} is analysis with fixed value of 'h', summary{2} is analysis with 'h' fitted; summary{:}.adt contain the following subfields
%
%         - params (struct): params.a and params.h are the Dirichlet parameters
%         - apriori_vals (float): a priori value of the index
%         - ah_llr (float): log likelihood ratio for the Dirichlet fit to the choice probability distribution
%         - thr_type (cell 1-D array): statistics for threshold type 1 (min), 2 (max), 3 (avg)
%
%             - tally_table (int 2-D array): tally_table(ithr,[1 2 3]) is the threshold, count of triads, count of trials
%             - means_per_set_adj (float 2-D array): means_per_set_adj(ithr,[1 2 3]) is the mean index for each threshold type
%             - eb_stds (float 2-D array): 1 s.d. error bar size
%             - frac_keep_list (float 1-d array): fraction of triplets to keep
%             - thr_ptr_use (int 1-D array): pointers into thresholds (tally_table(:,1)), corresponding to values in frac_keep_list
%
%     - global (struct): detailed analysis for addtree index, based on Dirichlet fits to choice probabilities for all triadic judgments, with fields
%
%         - a (int 3-D array): a(1,1:2,ih) is the fitted Dirichlet shape parameter 'a' and log likelihood per trial assuming h=h_fixlist(ih)
%         - ah (int 2-D array): ah(1,1:3) are the jointly fitted Dirichlet parameters 'a' and 'h' and log likelihood per trial for threshold value dirichlet.tallies(ithr,1)
%         - adt_hfixed (cell 2-D array): adt_hfixed{isv,ithr_type}(ithr,:,ih) is the sum (isv=1) or the variance (isv=2) of the addtree index, for threshold type ithr_type, threshold value dirichlet.tallies(ithr,1), assuming h=h_fixlist(ih)
%         - adt (cell 2-D array): adt{isv,ithr_type}(ithr,:) is the sum (isv=1) or the variance (isv=2) of the addtree index, for threshold type ithr_type, threshold value dirichlet.tallies(ithr,1), with 'a' and 'h' jointly fitted
% 
%     - private (struct): detailed analysis for addtree index, based on Dirichlet fits only to choice probabilities that meet the threshold criterion, with fields
%
%         - a (cell 1-D array): a{ithr_type}(ithr,1:2,ih) are the fitted Dirichlet shape parameter 'a' and log likelihood per trial for threshold type and threshold value dirichlet.tallies(ithr,1), assuming h=h_fixlist(ih)
%         - ah (cell 1-D array): ah{ithr_type}(ithr,1:3,ih) are the jointly fitted Dirichlet parameters 'a' and 'h' and log likelihood per trial for threshold type ithr_type and threshold value dirichlet.tallies(ithr,1)
%         - adt_hfixed (cell 2-D array): adt_hfixed{isv,ithr_type}(ithr,:,ih) is the sum (isv=1) or the variance (isv=2) of the addtree index, for threshold type ithr_type, threshold value dirichlet.tallies(ithr,1), assuming h=h_fixlist(ih)
%         - adt (cell 2-D array): adt{isv,ithr_type}(ithr,:) is the sum (isv=1) or the variance (isv=2) of the addtree index, for threshold type ithr_type, threshold value dirichlet.tallies(ithr,1), with 'a' and 'h' jointly fitted
% 
%     - **Dirichlet fits**
%     - dirichlet (struct): Dirichlet fits for lowest and highest thresholds for number of trials in a triad, with fields
% 
%         - tallies (int 2-D array): tallies(:,1) is threshold number of trials in a triad; tallies(:,2) is number of triads meeting the threshold; tallies(:,3) is number of trials in those triads
%         - columns_tallies (cell 1-D array): labels for columns of tallies
%         - h_fixlist (float 1-D array): list of values assumed for the discrete component, first element is 0
%         - a (int 3-D array): a(ithr,1,ih) is fitted value of Dirichlet shape parameter 'a' for triads meeting threshold of tallies(ithr,1) and assuming h=h_fixlist(ih); a(ithr,2,ih) is corresponding log likelihood per trial
%         - columns_a (cell 1-D array): labels for columns of a
%         - ah (int 2-D array): ah(ithr,1:2) are jointly fitted values of Dirichlet shape parameter 'a' and discrete parameter 'h' for triads meeting threshold of tallies(ithr,1); ah(ithr,3) is corresponding log likelihoood per trial
%         - columns_ah (cell 1-D array): labels for columns of ah
% 
%     - meta (struct): labels for dimensions of the variables in ad.global and ad.private
%
%     - counts (struct): summary of the input data, a structure with fields
%
%         - ntrials_found (int): number of individual judgments
%         - ntriads_found (int): number of distinct triads judged
%         - nstims_found (int): number of different stimuli
%         - unique_stims (int 1-D array): list of unique stimuli
%
%     - tallies (cell 1-D array): summary of the data used for each calculation in global and private, where tallies{ithr_type}(ithr,:) is [threshold pointer, number of triplets used, number of trials used] for threshold type ithr_type (1: min, 2: max, 3: avg, see note below regarding thresholds); the threshold value corresponding to ithr is dirichlet.tallies(ithr,1)
%
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_addtree (struct): aux,opts_addtree with defaults and values used
%     - opts_check (struct): aux.opts_check, with defaults filled in
%     - opts_dirfit_a (struct): options used for `rs_dirfit_choicedata` for fitting Dirichlet parameter 'a'
%     - opts_dirfit_ah (struct): options used for `rs_dirfit_choicedata` for fitting Dirichlet parameters 'a' and 'h'
%     - opts_triplike (struct): options used for `psg_umi_triplike`
%     - fig_handles (cell 1-D array): handles to figures for summary plot ((present only if if_plot>=1)
%     - fig_handle_detailed (handle): handle to figure for detailed plot (present only if if_plot=2)
%
% Note: Note regarding thresholds and global vs. private analyses
%     - Triplets are screened by a threshold criterion based on the number of trials before inclusion in the calculation of the symmetry and ultrametric indices.
%     - The criterion is applied three ways:  to the minimum numnber of trials of the three triads in a triplet, the maximum number, and the average number
%     - For the 'global' analysis, the selected triplets are used to calculate the symmetry and ultrametric indices, but all triads are used to calculate the Dirichlet parameters
%     - For the 'private' analysis, the selected triplets are used to calculate the symmetry and ultrametric indices and also to calculate the Dirichlet parameters 
%     - The 'private' analysis is substantially slower than the 'global' analysis, and values are of the indices are typically similar; it is only enabled by setting opts_addtree.if_private=1
% 
% Note: Triads, trials, triplets, and tents
%     - A triad is a set of three stimuli used in a triadic judgment: one stimulus is the reference, and the other two are the comparison stimuli
%     - A trial is a single judgment of similarity for a given triad
%     - A triplet is a set of three triads built out of the same three stimuli, in which each stimulus in turn serves as the reference
%     - The number of trials in a triplet is the sum of the number of trials in its three triads
%     - A tent is a set of six triads built out of four stimuli.  Three of the triads are a triplet built from the first three stimuil; the other three triads are triads built from the fourth stimulus and two of the first three.
% 
% See also: RS_DIRFIT_CHOICEDATA, RS_SYMUMI_CHOICEDATA, PSG_TRIPLET_CHOICES, LOGLIK_BETA_DISCRETE.
%
if (nargin<=1)
    aux=struct;
end
aux=filldefault(aux,'opts_addtree',struct);
aux.opts_addtree=filldefault(aux.opts_addtree,'if_log',1);
aux.opts_addtree=filldefault(aux.opts_addtree,'h_fixlist',[0 0.001 0.01 0.1]);
aux.opts_addtree=filldefault(aux.opts_addtree,'ntriplets_min',3);
aux.opts_addtree=filldefault(aux.opts_addtree,'ntents_min',3);
aux.opts_addtree=filldefault(aux.opts_addtree,'if_private',0);
%
aux.opts_addtree=filldefault(aux.opts_addtree,'if_frozen',1);
aux.opts_addtree=filldefault(aux.opts_addtree,'a_limits',[10^-2 10^2]);
aux.opts_addtree=filldefault(aux.opts_addtree,'a_optimset',struct());
aux.opts_addtree=filldefault(aux.opts_addtree,'ah_optimset',struct());
%
aux.opts_addtree=filldefault(aux.opts_addtree,'if_fast',1);
aux.opts_addtree=filldefault(aux.opts_addtree,'if_check',0);
aux.opts_addtree=filldefault(aux.opts_addtree,'if_tol',10^-5);
%
aux.opts_addtree=filldefault(aux.opts_addtree,'if_plot',1);
aux.opts_addtree=filldefault(aux.opts_addtree,'plot_label','ordinal analysis');
%
aux=filldefault(aux,'opts_check',struct);
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
aux=rs_aux_customize(aux,'rs_addtree_choicedata');
%
aux_out=struct;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
if isfield(aux.opts_addtree,'ad') 
    if aux.opts_addtree.if_plot>0
        [aux_out.fig_handles,aux_out.fig_handle_detailed,aux_out.summary]=rs_addtree_plot(data_comp,aux.opts_addtree.ad,aux);
    end
    ad=aux.opts_addtree.ad;
    return
end
dirfit_opts={'a_limits','a_optimset','ah_optimset','if_frozen','if_log'}; %options to transfer from aux.opts_addtree to aux_dirfit.opts_dirfit
triplike_opts={'if_fast','if_check','tol','if_vec','if_partition'}; %options to transfer from aux.opts_addtree to opts_triplike;
%
%set up random number generator
%
if_frozen=aux.opts_addtree.if_frozen;
if (if_frozen~=0) 
    rng('default');
    if (if_frozen<0)
        rand(1,abs(if_frozen));
    end
else
    rng('shuffle');
end
%
ad=struct;
%
% triadic?
%
if size(data_comp,2)~=5
    wmsg=sprintf('choice data must be triadic');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
%
choices=data_comp(:,end-1:end);
choices_nz=find(choices(:,2)>0);
choices_used=choices(choices_nz,:);
probs=choices_used(:,1)./choices_used(:,2);
choices_used=choices(choices_nz,:);
%
% enough data? at least two choices needed to fit a, three choices for a and h
%
nchoices=length(choices_nz);
choices_needed=3; %minimal choices needed
if nchoices<choices_needed
    wmsg=sprintf('insufficient choices available for fitting; at least %2.0f needed',choices_needed);
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
if all(or(probs==0,probs==1))
   wmsg=sprintf('all probabilities are 0 or 1; fitted parameter values are unreliable');
   aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
%
aux_out.opts_check=aux.opts_check;
aux_out.opts_addtree=aux.opts_addtree;
%
if aux_out.warn_bad>0
    disp('cannot proceed');
    disp(aux_out.warnings);
    return
end
%
ineq_logic_types={'exclude_addtree_trans','exclude_trans_tent'};
nineq=length(ineq_logic_types);
%
ncomps=6; %six rank choice probabilities to be compared
%
partitions=cell(0);
for ineq=1:nineq
    partitions{ineq}=psg_ineq_logic(ncomps,ineq_logic_types{ineq},setfield([],'if_log',aux.opts_addtree.if_log));
    if aux.opts_addtree.if_log
        disp(sprintf('created inequality logic for %s',ineq_logic_types{ineq}));
    end
end
permutes=psg_permutes_logic(ncomps,'flip_each');
nflips=size(permutes,2);
if aux.opts_addtree.if_log
    disp('creating permutation logic for surrogates')
    disp(sprintf(' size is %3.0f x %3.0f',size(permutes)));
end
%
%report number of stimulus types
%
data_nz=data_comp(choices_nz,:); %ignore choices with no trials
col_closer=4;
col_trials=5;
ntrials_found=sum(data_nz(:,col_trials));
ntriads_found=size(data_nz,1);
ustims_found=unique(reshape(data_nz(:,[1:3]),[3*ntriads_found,1]));
nstims_found=length(ustims_found);
%
ad.counts=struct;
ad.counts.ntrials_found=ntrials_found;
ad.counts.ntriads_found=ntriads_found;
ad.counts.nstims_found=nstims_found;
ad.counts.unique_stims_found=ustims_found;
%
nstims=nstims_found;
%
%make stimulus indices be consecutive from 1 to nstims
data=data_nz;
indices=data_nz(:,[1:3]);
[ui,ai,ci]=unique(indices(:));
data(:,[1:3])=reshape(ci,ntriads_found,3);
%
if aux.opts_addtree.if_log
    disp(sprintf('number of unique stimuli found: %3.0f; range from %3.0f to %3.0f',nstims_found,min(ustims_found),max(ustims_found)));
    disp(sprintf('number of trials found: %6.0f',ntrials_found));
    disp(sprintf('number of triads found: %6.0f',ntriads_found));
end
%
nt=3; %number of points in a triangle
triplets=nchoosek([1:nstims],nt); %triplets: unordered subsets of 3
ntriplets=nchoosek(nstims,nt); %ntriplets: number of unordered subsets of 3
%
% ncloser_triplets: [ntriplets,3]: N(d(a,b)<d(a,c)), N(d(b,c)<d(b,a)), N(d(c,a)<d(c,b))
% ntrials_triplets: [ntriplets,3]: total trials in above
[ncloser_triplets,ntrials_triplets,abc_list]=psg_triplet_choices(nstims,data); %extract triplets and sort
%
if aux.opts_addtree.if_log
    disp(sprintf('number of trials   after sorting: %6.0f',sum(ntrials_triplets(:)))) 
    disp(sprintf('number of triads   after sorting: %6.0f',sum(ntrials_triplets(:)>0)));
    disp(sprintf('number of triplets after sorting: %6.0f',sum(sum(ntrials_triplets,2)>0)));
    disp(sprintf('number of trials per triad range from %6.0f to %6.0f',min(ntrials_triplets(:)),max(ntrials_triplets(:))));
end
if (sum(ntrials_triplets(:))~=ntrials_found)
    wmsg=sprintf('mismatch of number of trials in data before sorting (%5.0f) vs after sorting (%5.0f)',ntrials_found,sum(ntrials_triplets(:)));
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
if aux.opts_addtree.if_log
    disp('creating tents from triplets');
end
%
%now create tents from the triplets
%
[ncloser,ntrials]=psg_tent_choices(nstims,data,ncloser_triplets,ntrials_triplets,aux.opts_addtree.if_log);
ntriplets_exclude=nchoosek(nstims-1,nt); %number of triplets that exclude a given stimulus
ntents=nstims*ntriplets_exclude;
%
h_fixlist=aux.opts_addtree.h_fixlist;
h_fixlist=unique([0 h_fixlist(:)']);
nhfix=length(h_fixlist);
%
ad.dirichlet=struct();
ad.dirichlet.columns_tallies={'min_trials_per_triad','ntriads','ntrials'};
ad.dirichlet.columns_a={'a','loglike_per_trial'};
ad.dirichlet.columns_ah={'a','h','loglike_per_trial'};
ad.dirichlet.h_fixlist=h_fixlist;
%
%Dirichlet fits, for fixed values of h and also h fitted
%code modified from psg_tentlike_demo, adapted for if_fixa=0, and rs_dirfit_choicedata
%
%Note that here and in private fits, values of a may differ slightly e.g., 0.01) from those obtained by psg_tentlike_demo
%This is because here, optimization uses loglik_beta_discrete, which adds a quadratic cost when h<0,
%while in psg_umi_triplike_demo, loptimization uses oglik_beta, which does not add a cost, but values of h<0 are replaced by the best fit with h>=0
%
aux_dirfit=struct;
for k=1:length(dirfit_opts)
    fn=dirfit_opts{k};
    if isfield(aux.opts_addtree,fn)
        aux_dirfit.opts_dirfit.(fn)=aux.opts_addtree.(fn);
    end
end
%
aux_dirfit.opts_dirfit.if_fit_a=0;
aux_dirfit.opts_dirfit.if_fit_h=0;
aux_dirfit.opts_dirfit.if_fit_ah=0;
aux_dirfit.opts_dirfit.fixed_h=0;
aux_dirfit.opts_dirfit.if_stats=0;
%
aux_dirfit.opts_check=aux.opts_check;
%
aux_dirfit_a=aux_dirfit;
aux_dirfit_a.opts_dirfit.if_fit_a=1;
%
aux_dirfit_ah=aux_dirfit;
aux_dirfit_ah.opts_dirfit.if_fit_ah=1;
%
aux_out.opts_dirfit_a=aux_dirfit_a.opts_dirfit;
aux_out.opts_dirfit_ah=aux_dirfit_ah.opts_dirfit;
%
% fit Dirichlet parameters according to occurrences in tents
% note that this weighs triplets more heavily if they occur in multiple tents
% ntrials_triplets is organized by triplets; ntrials is organized by tents, and trials are used more
%
ithr=0;
for thr=[min(ntrials(:)) max(ntrials(:))] %just compute extremes, and we only need min(ntrials(:)) for the index calculation below
    triads_use=find((ntrials(:)>=thr));
    ntriads_use=length(triads_use);
    ntrials_use=sum(ntrials(triads_use));
    if (ntriads_use>=aux.opts_addtree.ntriplets_min)
        ithr=ithr+1;
        ad.dirichlet.tallies(ithr,:)=[thr,ntriads_use,ntrials_use];
        data_use=[ncloser(triads_use) ntrials(triads_use)];
        if aux.opts_addtree.if_log
            disp(sprintf('fitting Dirichlet params after thresholding triads by %3.0f trials',thr))
        end
        %fixed  values of h
        for ihfix=1:nhfix
            %
            aux_dirfit_a.opts_dirfit.fixed_h=h_fixlist(ihfix);
            [dirfit_a,aux_dirfit_out_a]=rs_dirfit_choicedata(data_use,aux_dirfit_a);
            ad.dirichlet.a(ithr,:,ihfix)=[dirfit_a.a.val,dirfit_a.a.llnat_per_trial];
        end
        %fit a and h
        [dirfit_ah,aux_dirfit_out_ah]=rs_dirfit_choicedata(data_use,aux_dirfit_ah);
        ad.dirichlet.ah(ithr,:)=[dirfit_ah.ah.val',dirfit_ah.ah.llnat_per_trial];
    end
end
%
%analyze for consistency with addtree
%
ipg_strings={'private','global'};
npg=length(ipg_strings);
thr_types={'min','max','avg'};
nthr_types=length(thr_types);
%
%global analyses: Dirichlet fits not adjusted based on which triads are used
%private analyses: Dirichlet fits are private to the triads used
%
ad.meta=struct;
ad.global=struct;
ad.meta.thr_types=thr_types;
ad.meta.columns_tallies={'thr','ntriplets','ntrials'};
ad.meta.columns_a={'a','loglike_per_trial'}; %values of a and h determined from the selected trials
ad.meta.columns_ah={'a','h','loglike_per_trial'}; %values of a and h determined from the selected trials
ad.meta.columns_adt={'llr_addtree_trans_vs_trans_tent'}; %not excluded by addtree_trans vs not excluded by anything trans
ad.meta.columns_adt_hfixed=ad.meta.columns_adt; %like adt, but with fixed values of h
ad.meta.thr_types=thr_types;
ad.meta.ipg_strings=ipg_strings;
ad.meta.surr_types={'orig data','flip_all','flip_any'};
%
ad.global.a=ad.dirichlet.a(1,:,:); % values with h fixed
%compute using global a and h from unthresholded Dirichlet and save in r.ad.global.ah
if ad.dirichlet.ah(1,2)>=0 %use full fit if h>=0
    ad.global.ah=ad.dirichlet.ah(1,:);
else %otherwise use best fit with h=0
    ad.global.ah=[ad.dirichlet.a(1,1,1),0,ad.dirichlet.a(1,2,1)];
end
if aux.opts_addtree.if_private % compute these later using private a and h, to go in ad.private.[a|ah]{ithr_type}
    ad.private=struct;
    ad.private.a=cell(1,nthr_types); 
    ad.private.ah=cell(1,nthr_types);
end
%
ad.meta.global_private_d1={'mean of sum','variance of sum'};
ad.meta.global_private_d2={'threshold type'};
ipg_min=2-aux.opts_addtree.if_private;
for ipg=ipg_min:npg
    ad.(ipg_strings{ipg}).adt=cell(2,nthr_types);
    ad.(ipg_strings{ipg}).adt_hfixed=cell(2,nthr_types);
end
%
ad.meta.llr_d1={'threshold value'};
ad.meta.llr_d2=ad.meta.surr_types;
ad.meta.llr_d3={'hfixed'};
ad.meta.nsurr=length(ad.meta.surr_types);
nsurr=length(ad.meta.llr_d2); %three kinds of surrogates: native, flip all, flip any
%
llr_adt=cell(nsurr,2); %summed log likelihood ratio across trials, and summed variance of total 
llr_adt_hfixed=cell(nsurr,2);
llr_umi_hfixed=cell(nsurr,2);
surr_list={1,[1 nflips],[1:nflips]};
%
%if_fast=1: calculate probabilities for all triplets
ah=ad.global.ah;
ah_fixed=[squeeze(ad.dirichlet.a(1,1,:)),h_fixlist(:)];
obs_all=[reshape(ncloser',[ncomps 1 ntents]),reshape(ntrials',[ncomps 1 ntents])];
params.a=ah(1);
params.h=ah(2);
liks_all=psg_ineq_apply(params,obs_all,partitions,permutes);
if aux.opts_addtree.if_log
    disp(sprintf('preliminary global calculations done for a fitted at %6.4f, h fitted at %6.4f',params.a,params.h));
end   
liks_hfixed_all=zeros(nineq,nflips,ntents,nhfix);
for ihfix=1:nhfix
    params.a=ah_fixed(ihfix,1);
    params.h=ah_fixed(ihfix,2);
    liks_hfixed_all(:,:,:,ihfix)=psg_ineq_apply(params,obs_all,partitions,permutes);
    if aux.opts_addtree.if_log
        disp(sprintf('preliminary global calculations done for a fitted at %6.4f, h  fixed at %6.4f',params.a,params.h));
    end   
end
%
for ipg=ipg_min:2 %private and global, code modified from psg_umi_triplike_demo with nconform=0, if_fast=1
    if aux.opts_addtree.if_log
        disp(sprintf('%10s calculations',ipg_strings{ipg}));
    end
    for ithr_type=1:nthr_types %three kinds of thresholds: min, max, average
        if_ok=1;
        thr=0; %threshold
        ithr=1; %threshold pointer
        if aux.opts_addtree.if_log
            disp(sprintf('analyzing addtree likelihood ratio for threshold type %s',thr_types{ithr_type}));
        end
        nuse_prev=-1; %will allow for reuse if increasing the threshold doesn't change the number of triplets/tents used
        while (if_ok)
            switch thr_types{ithr_type}
                case 'min'
                    tents_use=find(min(ntrials,[],2)>=thr);
                    thr_val=thr;
                case 'max'
                    tents_use=find(max(ntrials,[],2)>=thr);
                    thr_val=thr;
                case 'avg'
                    tents_use=find(sum(ntrials,2)>=thr);
                    thr_val=thr/ncomps; %average not total
            end
            if (length(tents_use)>=aux.opts_addtree.ntents_min)
                ntents_use=length(tents_use);
                if ntents_use~=nuse_prev
                    did_or_skipped='did'; %have to calculate
                    nuse_prev=ntents_use;
                    ntrials_use=sum(sum(ntrials(tents_use,:)));
                    ad.tallies{ithr_type}(ithr,:)=[thr_val ntents_use ntrials_use]; %threshold, number of tents, number of trials
                    %compute private best-fitting a and h
                    if (ipg==1)
                        data_use=[reshape(ncloser(tents_use,:),ncomps*ntents_use,1) reshape(ntrials(tents_use,:),ncomps*ntents_use,1)];
                        %private fits, assuming fixed values of h
                        for ihfix=1:nhfix
                            aux_dirfit_a.opts_dirfit.fixed_h=h_fixlist(ihfix);
                            [dirfit_a,aux_dirfit_out_a]=rs_dirfit_choicedata(data_use,aux_dirfit_a);
                            ad.private.a{ithr_type}(ithr,:,ihfix)=[dirfit_a.a.val,dirfit_a.a.llnat_per_trial];
                        end
                        %private fit for a and h
                        [dirfit_ah,aux_dirfit_out_ah]=rs_dirfit_choicedata(data_use,aux_dirfit_ah);
                        if dirfit_ah.ah.val(2)>=0 %ensure h >=0
                            ad.private.ah{ithr_type}(ithr,:)=[dirfit_ah.ah.val',dirfit_ah.ah.llnat_per_trial];
                        else
                            ad.private.ah{ithr_type}(ithr,:)=[dirfit_a.a.val,0,dirfit_ah.ah.llnat_per_trial];
                        end
                    end
                    liks=zeros(nineq,nflips,ntents_use);
                    liks_hfixed=zeros(nineq,nflips,ntents_use,nhfix);
                    %
                    %fast global option:calculate probabilities for all triplets and later select
                    %
                    if ipg==2
                        liks=liks_all(:,:,tents_use); %d1: addtree_trans vs trans_tent, d2: flips, d3: tent
                        liks_hfixed=liks_hfixed_all(:,:,tents_use,:); %d1: addtree_trans vs trans_tent, d2: flips, d3: tent, d4:h_fixed
                    else % ipg=1 (private)
                        ah=ad.private.ah{ithr_type}(ithr,:); %a and h both fitted
                        ah_fixed=[squeeze(ad.private.a{ithr_type}(ithr,1,:)),h_fixlist(:)]; %a fitted, h fixed
                        params.a=ah(1);
                        params.h=ah(2);
                        liks=psg_ineq_apply(params,obs_all(:,:,tents_use),partitions,permutes);
                        for ihfix=1:nhfix
                            params.a=ah_fixed(ihfix,1);
                            params.h=ah_fixed(ihfix,2);
                            liks_hfixed(:,:,:,ihfix)=psg_ineq_apply(params,obs_all(:,:,tents_use),partitions,permutes);
                        end
                    end %ipg
                    %do statistics
                    %likelihood of addtree and trans,/likelihood(trans), i.e., exclude_addtree_trans/exclude_trans_tent';
                    %dimensions reordered to match those of loglik_rat_sym|umi[|_hfixed] of psg_umi_triplike_demo
                    loglikrats=transpose(log(reshape(liks(1,:,:)./liks(2,:,:),[nflips ntents_use]))); %after transpose: d1: tents, d2: flips
                    loglikrats_hfixed=permute(log(reshape(liks_hfixed(1,:,:,:)./liks_hfixed(2,:,:,:),[nflips ntents_use nhfix])),[2 1 3]); %d1:tents, d2:flips, d3:h
                    for isurr=1:nsurr %for each kind of surrogate
                        surr_sel=surr_list{isurr}; %for isurr=1, this is just the original data (1)
                        llr_adt{isurr,1}=sum(mean(loglikrats(:,surr_sel),2),1);
                        llr_adt_hfixed{isurr,1}=reshape(sum(mean(loglikrats_hfixed(:,surr_sel,:),2),1),[1 1 nhfix]);
                        if (isurr>1)
                            %each tent contributes independently to the variance
                            %variance for each tent is normalized by N not N-1, since we have all the values
                            llr_adt{isurr,2}=sum(var(loglikrats(:,surr_sel),1,2),1);
                            llr_adt_hfixed{isurr,2}=reshape(sum(var(loglikrats_hfixed(:,surr_sel,:),1,2),1),[1 1 nhfix]);
                        else %isurr=1: original data. Here, goal is for psg_umi_triplike_plota to compute standard error of the mean
                            %which is sqrt(var)/ntents_use, but
                            %psg_umi_triplike_plota will find square root and then divide by ntents_use
                            %so here we just compute var, normalized by N-1 since it is a sample
                            %here, surr_sel=1
                            llr_adt{isurr,2}=var(loglikrats(:,surr_sel),0,1);
                            llr_adt_hfixed{isurr,2}=reshape(var(loglikrats_hfixed(:,surr_sel,:),0,1),[1 1 nhfix]);
                        end
                        for isv=1:2% mean and variance
                            ad.(ipg_strings{ipg}).adt{isv,ithr_type}(ithr,isurr)=llr_adt{isurr,isv};
                            ad.(ipg_strings{ipg}).adt_hfixed{isv,ithr_type}(ithr,isurr,:)=llr_adt_hfixed{isurr,isv};
                        end %isv
                     end %isurr
                else %change in threshold does not change which triplets are included
                     did_or_skipped='skp';
                     ad.tallies{ithr_type}(ithr,:)=ad.tallies{ithr_type}(ithr-1,:);
                     ad.tallies{ithr_type}(ithr,1)=thr_val; %threshold is new
                     if (ipg==1)
                        ad.private.a{ithr_type}(ithr,:,:)=ad.private.a{ithr_type}(ithr-1,:,:);
                        ad.private.ah{ithr_type}(ithr,:)=ad.private.ah{ithr_type}(ithr-1,:);
                     end
                     for isurr=1:nsurr
                         for isv=1:2% mean and variance
                             ad.(ipg_strings{ipg}).adt{isv,ithr_type}(ithr,isurr)=llr_adt{isurr,isv};
                             ad.(ipg_strings{ipg}).adt_hfixed{isv,ithr_type}(ithr,isurr,:)=llr_adt_hfixed{isurr,isv};
                         end %isv
                     end %isurr
                 end %nuse_prev
                 if aux.opts_addtree.if_log
                     disp(sprintf('%s ipg %3.0f ithr_type %3.0f ithr %3.0f thr %3.0f ntents_use %6.0f size(liks) %4.0f %4.0f %6.0f size(liks_hfixed) %4.0f %4.0f %6.0f %4.0f',...
                         did_or_skipped,ipg,ithr_type,ithr,thr,ntents_use,size(liks),size(liks_hfixed)));
                 end
                 thr=thr+1; %threshold
                 ithr=ithr+1; %pointer
             else
                 if_ok=0;
             end
         end %if_ok
    end %thr_type
end %ipg
%
[aux_out.fig_handles,aux_out.fig_handle_detailed,su.summary]=rs_addtree_plot(data_comp,ad,aux);
return
end

`rs_align_coordsets(data_in, aux)`

[data_out,aux_out]=rs_align_coordsets(data_in,aux) aligns a dataset structure with partially overlapping stimuli

Each of the records in the dataset structure data_in contains the responses to one or more stimuli, with stimulus identity in record k determined by the strings in data_in.sas{k}.typenames. The stimuli in each of the records may differ, and may overlap.

Each of the corresponding records in the dataset structure data_out has a stimulus set equal to the union of all of the stimulus sets in the records of data_in. For a stimulus in the kth record of data_out for which there is no entry in data_in, the coordinates are NaN. See note below regarding stimulus coordinates.

The stimulus labels in data_out.sas{k}.typenames are in alphabetical order, and are identical for all of the records Thus, even if there is complete overlap between the stimuli in data_in, the dataset structure data_out may differ.

The stimulus coordinates in data_out.sas{k} are correspondingy aligned; see note below regarding stimulus coordinates.

Input arguments:

data_in (struct) –
dataset structure to be aligned containing n records, with fields
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record
aux (struct) –
auxiliary options, may be omitted, with fields
- opts_align (struct) – options for alignment and consistency checking, with fields
  - if_log (int) – 1 to log progress, 0 to suppress; default is 1
  - min (int or char) – minimum number of datasets that must contain a stimulus, in order for the stimulus to be included in data_out; default is 1, equivalent to 'any'; can also be 'all', meaning that stimuli must be present in all datasets to be included in data_out
  - if_type_coords_remake (int) – controls alignment of stimulus coordinates, typically omitted or set to [], see note below regarding stimulus coordinates
  - if_btcremz (int) – typically omitted, defaults to 1, see note below regarding labels for binary texture coordinates
- opts_check (struct) – options for consistency checking, with field
  - if_warn (int) – 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
- opts_rays (struct) – options for rays, typically omitted; see note below regarding rays

Output arguments:

data_out ( struct ) –

aligned dataset structure with n records, same format as as data_in
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_align (struct) – aux.opts_align, with defaults filled in, and also
  - ovlp_array_all (integer array) – the overlap array for the pooled dataset. Each row corresponds to one of the typenames (see sa_pooled below) contained in any of the input records; ovlp_array_all(s,k)=1 if the stmulus is present in record k of data_in
- opts_check (struct) – aux.opts_check, with defaults filled in
- opts_import (struct) – aux.opts_import, with defaults filled in
- opts_rays (cell array) – opts_rays{k} is a structure which contains the options used for creating rays in record k in data_out
- ovlp_array (int 2-D array) – ovlp_array(s,k)=1 if the stimulus data_out.sets{:}.typenames{s} is present in record k of data_in, 0 otherwise
- sa_pooled (struct) – the stimulus metadata structure for the pooled stimulus set; see note below regarding stimulus coordinates
- rayss (cell array of struct) – rayss{k} is the ray structure for record k in data_out; see note below regarding rays
- opts_btcremz (cell array of struct) – see note below regarding labels for binary texture coordinates

General notes

For all records with data_in.sets{k}.type='data', the strings in data_in.sets{k}.paradigm_type must agree.
Pipeline: data_out.sets{k}.pipeline.sets{1} contains metadata for the kth record of data_in; data_out.sets{k}.pipeline.sets_combined{:} contains metadata from all records of data_in; data_out.sets{1}.pipeline.type='align'.

Note regarding stimulus coordinates

Stimulus coordinates are optionally present in data_in.sas{k} in the fields type_coords, btc_specoords, or btc_augcoords. Their treatment is governed by aux.opts_align.if_type_coords_remake. If this field is omitted or [], behavior is determined by the stimulus coordinates supplied in data_in.sas{k}. A value of 0 attempts to merge the coordinates, a value of 1 does not.
Merging (if_type_coords_remake=0, or, stimulus coordinates are not all 0 or 1, or a non-square array with at least 2 columns): In the aligned stimulus metadata structure data_out.sas{k}, if the stimulus typename is not present in data_in.sas{k}, stimulus coordinates are NaN. In the pooled stimulus metadata structure aux_out.sa_pooled, stimulus coordinates are taken from the first occurence of the typename in data_in.sas{:} is used. A similar behavior is applied to other subfields of the stimulus metadata structure that relate to individual stimuli. These fields are listed in aux_out.opts_align.fields_align, which defaults in psg_align_coordsets to {'specs' 'spec_labels' 'btc_augcoords' 'btc_specoords' 'type_coords'}
No merging (if_type_coords_remake=1, or stimulus coordinates absent, or, stimulus coordinates do not meet above requirements): In the aligned stimulus metadata structure data_out.sas{k}, dummy coordinates are created in one of the fields type_coords, btc_specoords, or btc_augcoords (the field chosen is determined by the first of that list that is found in data_in.sas{:}). The dummy coordinates can be an empty matrix, zeros, ones, or the identity, as determined by aux.opts_import.type_coords_def. This defaults to 'none', can be 'zeros','ones', or 'eye', and the value used is reported in aux_out.opts_import.type_coords_def and aux_out.opts_align.type_coords_def.
The behavior taken is reported in aux_out.opts_align.if_type_coords_remake.

Note regarding rays

The ray structure describes relationships among the simulus coordinates: rays, i.e., sets of stimuli that lie along an axis or a ray from the origin, rings, stimuli that lie at an approximately equal distance from the origin, and nearest neighbors. It is only created if there is a valid set of stimulus coordinates.

Note regarding labels for binary texture coordinates

if_btcremz is only relevant for binary texture domain metadata, and controls whether an attempt should be made to simplify fields sas{k}.spec_labels and sas{k}.typenames prior to matching and alignmnent. If 1 (default), coordinates that are specified as zero are removed, provided that this does not change the stimulus after maximum-entropy extension. For example, in data_in.sas{k}.spec_labels, 'b=-0.00 c=-0.40' becomes 'c=-0.40' the corresponding entry in data_in.sas{k}.typenames, 'bm0000cm0400' becomes 'cm0400'. An all-zero coordinate becomes 'rand'. Results for simplification of record k are returned in opts_btcremz{k}.

See also

rs_aux_customize, rs_findrays, PSG_ALIGN_COORDSETS, rs_import_coordsets, PSG_COORD_PIPE_UTIL, PSG_BTCREMZ, rs_check_coordsets.

Source code in src/rs_align_coordsets.m

function [data_out,aux_out]=rs_align_coordsets(data_in,aux)
% [data_out,aux_out]=rs_align_coordsets(data_in,aux)
% aligns a `dataset structure` with partially overlapping stimuli
%
% Each of the records in the `dataset structure` `data_in` contains the responses to one or more stimuli,
% with stimulus identity in record k determined by the strings in data_in.sas{k}.typenames. The stimuli
% in each of the records may differ, and may overlap.
%
% Each of the corresponding records in the `dataset structure` data_out has a stimulus set 
% equal to the union of all of the stimulus sets in the records of `data_in`.  For a stimulus in the kth record of
% `data_out` for which there is no entry in `data_in`, the coordinates are NaN. See note below regarding stimulus coordinates.
%
% The stimulus labels in data_out.sas{k}.typenames are in alphabetical order, and are identical for all of the records
% Thus, even if there is complete overlap between the stimuli in `data_in`, the `dataset structure` `data_out` may differ.
%
% The stimulus coordinates in data_out.sas{k} are correspondingy aligned; see note below regarding stimulus coordinates.
%
% Args:
%   data_in (struct): `dataset structure` to be aligned containing n records, with fields
%
%     - ds (cell array): `coordinate structure`, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
%     - sas (cell array): `stimulus metadata structure`, sas{k} is the stimulus metadata for the kth record
%     - sets (cell array): `set metadata structure`, sets{k} is the response metadata for the kth record
% 
%   aux (struct): auxiliary options, may be omitted, with fields
%
%     - opts_align (struct): options for alignment and consistency checking, with fields
%
%         - if_log (int): 1 to log progress, 0 to suppress; default is 1
%         - min (int or char): minimum number of datasets that must contain a stimulus, in order for the stimulus to be included in data_out;
%         default is 1, equivalent to 'any';  can also be 'all', meaning that stimuli must be present in all datasets to be included in data_out
%         - if_type_coords_remake (int):  controls alignment of stimulus coordinates, typically omitted or set to [], see note below regarding stimulus coordinates
%         - if_btcremz (int): typically omitted, defaults to 1, see note below regarding labels for binary texture coordinates
%
%     - opts_check (struct): options for consistency checking, with field
%
%         - if_warn (int): 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
%
%     - opts_rays (struct): options for rays, typically omitted; see note below regarding rays
%
% Returns:
%   data_out (struct): aligned `dataset structure` with n records, same format as  as `data_in`
%
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_align (struct): aux.opts_align, with defaults filled in, and also
%
%         - ovlp_array_all (integer array): the overlap array for the pooled dataset.  Each row corresponds to one of the 
%         typenames (see sa_pooled below) contained in any of the input records; ovlp_array_all(s,k)=1 if the stmulus is present in record
%         k of `data_in`
%
%     - opts_check (struct): aux.opts_check, with defaults filled in
%     - opts_import (struct): aux.opts_import, with defaults filled in
%     - opts_rays (cell array): opts_rays{k} is a structure which contains the options used for creating rays in record k in `data_out`
%     - ovlp_array (int 2-D array): ovlp_array(s,k)=1 if the stimulus data_out.sets{:}.typenames{s} is present in record k of `data_in`, 0 otherwise
%     - sa_pooled (struct): the `stimulus metadata structure` for the pooled stimulus set; see note below regarding stimulus coordinates
%     - rayss (cell array of struct): rayss{k} is the `ray structure` for record k in `data_out`; see note below regarding rays
%     - opts_btcremz (cell array of struct): see note below regarding labels for binary texture coordinates
%
% Note: General notes
%     - For all records with data_in.sets{k}.type='data', the strings in data_in.sets{k}.paradigm_type must agree.
%     - Pipeline: data_out.sets{k}.pipeline.sets{1} contains metadata for the kth record of `data_in`;
%     data_out.sets{k}.pipeline.sets_combined{:} contains metadata from all records of `data_in`;
%     data_out.sets{1}.pipeline.type='align'.
%
% Note: Note regarding stimulus coordinates
%     - Stimulus coordinates are optionally present in data_in.sas{k} in the fields type_coords, btc_specoords, or btc_augcoords.
%     Their treatment is governed by aux.opts_align.if_type_coords_remake.
%     If this field is omitted or [], behavior is determined by the stimulus coordinates supplied in data_in.sas{k}.
%     A value of 0 attempts to merge the coordinates, a value of 1 does not.
%
%     - Merging (if_type_coords_remake=0, or, stimulus coordinates are not all 0 or 1, or a non-square array with at least 2 columns):
%     In the aligned `stimulus metadata structure` data_out.sas{k}, if the stimulus typename is not present in data_in.sas{k}, stimulus coordinates are NaN.
%     In the pooled `stimulus metadata structure` aux_out.sa_pooled, stimulus coordinates are taken from the first occurence of the typename in data_in.sas{:} is used.
%     A similar behavior is applied to other subfields of the stimulus
%     metadata structure that relate to individual stimuli.  These fields are listed in aux_out.opts_align.fields_align, which defaults in psg_align_coordsets
%     to {'specs'  'spec_labels'  'btc_augcoords'  'btc_specoords'  'type_coords'}
%
%     - No merging (if_type_coords_remake=1, or stimulus coordinates absent, or, stimulus coordinates do not meet above requirements):
%     In the aligned `stimulus metadata structure` data_out.sas{k}, dummy coordinates are created in one of the fields
%     type_coords, btc_specoords, or btc_augcoords (the field chosen is determined by the first of that list that is found in data_in.sas{:}).
%     The dummy coordinates can be an empty matrix, zeros, ones, or the identity,
%     as determined by aux.opts_import.type_coords_def.  This defaults to 'none', can be 'zeros','ones', or 'eye', and the value used is
%     reported in aux_out.opts_import.type_coords_def and aux_out.opts_align.type_coords_def.
% 
%     - The behavior taken is reported in aux_out.opts_align.if_type_coords_remake.
%
% Note: Note regarding rays
%     - The `ray structure` describes relationships among the simulus coordinates: 
%     `rays`, i.e., sets of stimuli that lie along an axis or a ray from the origin,
%     `rings`, stimuli that lie at an approximately equal distance from the origin, and nearest neighbors.
%     It is only created if there is a valid set of stimulus coordinates.  
%
% Note: Note regarding labels for binary texture coordinates
%     - if_btcremz is only relevant for `binary texture domain` metadata, and controls whether an attempt should be made to simplify fields sas{k}.spec_labels and sas{k}.typenames
%     prior to matching and alignmnent.  If 1 (default), coordinates that are specified as zero are removed, provided
%     that this does not change the stimulus after maximum-entropy extension.  For example, in data_in.sas{k}.spec_labels, 'b=-0.00 c=-0.40' becomes 'c=-0.40'
%     the corresponding entry in data_in.sas{k}.typenames, 'bm0000cm0400' becomes 'cm0400'.  An
%     all-zero coordinate becomes 'rand'. Results for simplification of record k are returned in opts_btcremz{k}.
% 
% See also:
%   RS_AUX_CUSTOMIZE, RS_FINDRAYS, PSG_ALIGN_COORDSETS, RS_IMPORT_COORDSETS, PSG_COORD_PIPE_UTIL, PSG_BTCREMZ, RS_CHECK_COORDSETS.
%
if (nargin<=1)
    aux=struct;
end
%
aux=filldefault(aux,'opts_align',struct);
aux.opts_align=filldefault(aux.opts_align,'if_log',1);
aux.opts_align=filldefault(aux.opts_align,'min',1);
aux.opts_align=filldefault(aux.opts_align,'if_btcremz',1);
%
aux=filldefault(aux,'opts_check',struct);
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
aux=filldefault(aux,'opts_rays',struct);
aux=filldefault(aux,'opts_import',struct); %alignment needs to know how to set up type_coords if none are supplied
%
aux=rs_aux_customize(aux,'rs_align_coordsets');
%
data_out=struct;
aux_out=struct;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
%
if isnumeric(aux.opts_align.min)
    min_string=sprintf('%1.0f',aux.opts_align.min);
else
    min_string=aux.opts_align.min;
end
%
nsets=length(data_in.sets);
%
%check internal consistency
%
for iset=1:nsets
    data_check=struct;
    data_check.ds{1}=data_in.ds{iset};
    data_check.sas{1}=data_in.sas{iset};
    data_check.sets{1}=data_in.sets{iset};
    check=rs_check_coordsets(data_check,setfield(aux.opts_check,'set_num_offset',iset-1));
    if ~isempty(check.warnings) %since strvcat([],[])~=[]       
        aux_out.warnings=strvcat(aux_out.warnings,check.warnings);
        warn_leadin=getfield(getfield(rs_aux_customize(struct()),'overall'),'warn_leadin');
        for k=1:size(aux_out.warnings,1)
            disp(cat(2,warn_leadin,aux_out.warnings(k,:)));
        end
    end
    aux_out.warn_bad=aux_out.warn_bad+check.warn_bad;
end
%
%check that data paradigms are the same (model paradigms may differ)
paradigm_types=cell(1,nsets);
types=cell(1,nsets);
paradigm_type=[];
paradigm_match=1;
for iset=1:nsets
    types{iset}=data_in.sets{iset}.type;
    data_in.sets{iset}=filldefault(data_in.sets{iset},'paradigm_type','unrecognized');
    paradigm_types{iset}=data_in.sets{iset}.paradigm_type;
    if strcmp(types{iset},'data')
        if isempty(paradigm_type)
            paradigm_type=paradigm_types{iset};
        else
            if ~strcmp(paradigm_types{iset},paradigm_type)
                paradigm_match=0;
            end
        end
    end
    if strcmp(paradigm_type,'btc')
        if aux.opts_align.if_btcremz
            [sas_new,change_list,opts_btcremz_used]=psg_btcremz(data_in.sas{iset});
            aux_out.opts_btcremz{iset}=opts_btcremz_used;
            if aux.opts_align.if_log
                disp(sprintf(' set %3.0f: simplification attempted, %3.0f coords simplified (label: %s)',iset,length(change_list),data_in.sets{iset}.label));
            end
            if length(change_list)>0
                if aux.opts_align.if_log
                    for k=1:length(change_list)
                        kch=change_list(k);
                        disp(sprintf('%15s -> %15s',data_in.sas{iset}.typenames{kch},sas_new.typenames{kch}));
                    end
                end
                data_in.sas{iset}=sas_new;
            end
        end
    end %paradigm type=btc
end
if paradigm_match==0
    wmsg=sprintf('paradigm types disagree');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
    if aux.opts_align.if_log
        disp([types; paradigm_types])
    end
end
aux_out.opts_check=aux.opts_check;
aux_out.opts_import=aux.opts_import;
if aux_out.warn_bad==0
    paradigm_type=paradigm_types{1};
    if aux.opts_align.if_log
        disp(sprintf('proceeding with alignment of %3.0f datasets, paradigm type %s, stimuli must be present in %s',nsets,paradigm_type,min_string));
    end
    %alignment needs to know how to set up type_coords if none are supplied
    [sets_align,ds_align,sas_align,ovlp_array,sa_pooled,opts_align_used]=...
        psg_align_coordsets(data_in.sets,data_in.ds,data_in.sas,setfield(aux.opts_align,'type_coords_def',aux.opts_import.type_coords_def));
    %
    for iset=1:nsets
        [rays,wmsg,opts_rays_used]=rs_findrays(sas_align{iset},sets_align{iset}.label,aux.opts_rays);
        if ~isempty(wmsg)
            aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
        end
        aux_out.opts_rays{iset}=opts_rays_used;
        aux_out.rayss{iset}=rays;
    end
    data_out.ds=ds_align;
    data_out.sas=sas_align;
    data_out.sets=sets_align;
    data_out.sets{1}.type=data_in.sets{1}.type;
    aux_out.ovlp_array=ovlp_array;
    aux_out.sa_pooled=sa_pooled;
    aux_out.opts_align=opts_align_used;
    %
    pipeline_opts=struct;
    pipeline_opts.opts_align=opts_align_used;
    for iset=1:nsets
        data_out.sets{iset}.pipeline=psg_coord_pipe_util('align',pipeline_opts,data_in.sets{iset},[],data_in.sets);
    end
else
    disp('cannot proceed');
    disp(aux_out.warnings);
end
return
end

`rs_aux_customize(aux, caller, aux_default_filename)`

aux_out=rs_aux_customize(aux,caller,aux_default_filename) sets the auxiliary inputs to customized defaults

This read a file created by rs_aux_defaults_define at the time of installation, and should be edited to customize the default auxiliary inputs as needed.

Input arguments:

aux (struct) –

options structure, typically with subfields with names like opts_read, opts_disp, opts_knit, etc.
caller (char) –

name of calling function, e.g., 'rs_get_coordsets'; may be empty
aux_default_filename (char) –

full path to the file created by rs_aux_defaults_define; defaults to 'rs_aux_defaults.mat'

Output arguments:

aux_out ( struct ) –

aux, with defaults filled in

Default assignment precedence

First priority to value explicitly provided in a function call
Then, value specified in an rs_*.m module via a 'filldefault' statement
Then, value listed in the specific.(caller) section of rs_aux_defaults_define
Then, value listed in the generic section of rs_aux_defaults_define

Source code in src/rs_aux_customize.m

function aux_out=rs_aux_customize(aux,caller,aux_default_filename)
% aux_out=rs_aux_customize(aux,caller,aux_default_filename) sets the auxiliary inputs to customized defaults
%
% This read a file created by `rs_aux_defaults_define` at the time of installation, and should be edited to customize the default auxiliary inputs as needed.
%
% Args:
%   aux (struct): options structure, typically with subfields with names like opts_read, opts_disp, opts_knit, etc.
%
%   caller (char): name of calling function, e.g., 'rs_get_coordsets'; may be empty
%
%   aux_default_filename (char): full path to the file created by `rs_aux_defaults_define`; defaults to 'rs_aux_defaults.mat'
%
% Returns:
%   aux_out (struct): aux, with defaults filled in
%
% Notes: Default assignment precedence
%     - First priority to value explicitly provided in a function call
%     - Then, value specified in an rs_\*.m  module via a 'filldefault' statement
%     - Then, value listed in the specific.(caller) section of `rs_aux_defaults_define`
%     - Then, value listed in the generic section of `rs_aux_defaults_define`
%
% See also: RS_AUX_DEFAULTS_DEFINE, RS_AUX_FORCE.
%
if (nargin==0)
    aux=struct();
end
if isempty(aux)
    aux=struct();
end
if (nargin<=1)
    caller=[];
end
if (nargin<=2)
    aux_default_filename=[];
end
if isempty(aux_default_filename)
    aux_default_filename='rs_aux_defaults.mat';
end
s=load(aux_default_filename);
aux_fields=fieldnames(aux);
for ifn=1:length(aux_fields)
    fn=aux_fields{ifn}; %fn='opts_read' or similar
    %create the defaults from s.generic.(fn), overridden by s.specific.(caller).(fn)
    defaults=struct;
    if isfield(s.generic,fn)
        defaults=s.generic.(fn);
    end
    if ~isempty(caller)
        if isfield(s.specific,caller)
            if isfield(s.specific.(caller),fn)
                specific_fns=fieldnames(s.specific.(caller).(fn));
                for ifn=1:length(specific_fns)
                    sfn=specific_fns{ifn};
                    defaults.(sfn)=s.specific.(caller).(fn).(sfn);
                end
            end
        end
    end %overrides
    def_names=fieldnames(defaults);
    for id=1:length(def_names)
        aux.(fn)=filldefault(aux.(fn),def_names{id},defaults.(def_names{id}));
    end %default values to fill
end %fields of aux
aux_out=aux;
if isfield(s,'overall')
    aux_out.overall=s.overall;
end
return
end

`rs_aux_force(aux_name, caller, aux_force_filename, aux_default_filename)`

[opts_forced,def,force]=rs_aux_force(aux_name,caller,aux_force_filename,aux_default_filename) sets an options structure to override user-customized defaults

This module is solely intended for use during benchmarking with the rs_*test modules.

Input arguments:

aux_name (char) –

name of an options structure, e.g., opts_read, opts_disp
caller (char) –

name of calling function, e.g., 'rs_get_coordsets'; may be empty
aux_force_filename (char) –

full path to the overriding set of default auxiliary inputs
aux_default_filename (char) –

full path to the default auxiliary input file created by rs_aux_defaults_define; defaults to 'rs_aux_defaults.mat'

Output arguments:

opts_forced ( struct ) –

structure whose fields in aux_force_filename differ from those in aux_default_filename
def ( struct ) –

auxiliary inputs from aux_default_filename
force ( struct ) –

auxiliary inputs from aux_force_filename

Source code in src/rs_aux_force.m

function [opts_forced,def,force]=rs_aux_force(aux_name,caller,aux_force_filename,aux_default_filename)
% [opts_forced,def,force]=rs_aux_force(aux_name,caller,aux_force_filename,aux_default_filename) sets an options structure to override user-customized defaults
% 
% This module is solely intended for use during benchmarking with the rs_\*test modules.  
%
% Args:
%   aux_name (char): name of an options structure, e.g., opts_read, opts_disp
%
%   caller (char): name of calling function, e.g., 'rs_get_coordsets'; may be empty
%
%   aux_force_filename (char): full path to the overriding set of default auxiliary inputs
%
%   aux_default_filename (char): full path to the default auxiliary input file created by `rs_aux_defaults_define`; defaults to 'rs_aux_defaults.mat'
%
% Returns:
%   opts_forced (struct): structure whose fields in aux_force_filename differ from those in aux_default_filename
%   
%   def (struct): auxiliary inputs from aux_default_filename
%
%   force (struct): auxiliary inputs from aux_force_filename
%
% See also: RS_AUX_DEFAULTS_DEFINE, RS_AUX_CUSTOMIZE.
%
if (nargin<=3)
    aux_default_filename='rs_aux_defaults.mat';
end
opts_forced=struct;
def=rs_aux_customize(setfield(struct(),aux_name,struct()),caller,aux_default_filename);
force=rs_aux_customize(setfield(struct(),aux_name,struct()),caller,aux_force_filename);
fns=fieldnames(force.(aux_name));
for ifn=1:length(fns)
    fn=fns{ifn};
    if isfield(def.(aux_name),fn) %is this in the defaults?
        if ~isequal(def.(aux_name).(fn),force.(aux_name).(fn)) %does it match
            opts_forced.(fn)=force.(aux_name).(fn);
        end
    else
        opts_forced.(fn)=force.(aux_name).(fn);
    end
end
return
end

`rs_check_coordsets(data_in, opts)`

[check,opts_used]=rs_check_coordsets(data_in,opts) checks the consistency of dataset structures between and within records, and tallies the available dimensions and stimuli

Input arguments:

data_in (struct) –
dataset structure, with fields
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record
opts (struct) –
auxiliary options, may be omitted, with fields
- if_warn (int) – 1 to display warnings, 0 to omit; default is 0
- if_checkorder (int) – 1 to check that the order of the stimuli, as listed in ds.sas{k}.typenames, is the same, 0 to compare independent of odrer; default is 0
- record_num_offset (int) – value to add to the record number in displaying aarnings; default is 0

Output arguments:

check ( struct ) –
summary of the consistency check, with fields
- warnings (char) – warning messages
- warn_bad (int) – count of severe warnings that likely prevent processing
- nsets (int) – number of records
- nstims_each (int 1-D array) – nstims_each(k) is the number of stimuli in record k
- dim_list_each (cell array) – dim_list_each{k} is the list of dimensions available in record k
- dim_list_union (int 1-D array) – union of dim_list_each{:}
- dim_list_inter (int 1-D array) – intersection of dim_list_each{:}
- typenames_each (cell array) – typenames_each{k} is data_in.sas{k}.typenames, in original order
- typenames_union (cell array) – union of typenames_each{:}, alphabetized
- typenames_inter (cell array) – intersection of typenames_each{:}, alphabetized
opts_used ( struct ) –

opts, with defaults filled in

Source code in src/rs_check_coordsets.m

function [check,opts_used]=rs_check_coordsets(data_in,opts)
% [check,opts_used]=rs_check_coordsets(data_in,opts) checks the consistency of `dataset structures` between and within records,
% and tallies the available dimensions and stimuli
%
% Args:
%   data_in (struct): `dataset structure`, with fields
%
%     - ds (cell array): `coordinate structure`, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
%     - sas (cell array): `stimulus metadata structure`, sas{k} is the stimulus metadata for the kth record
%     - sets (cell array): `set metadata structure`, sets{k} is the response metadata for the kth record
%
%   opts (struct): auxiliary options, may be omitted, with fields
%
%     - if_warn (int): 1 to display warnings, 0 to omit; default is 0
%     - if_checkorder (int): 1 to check that the order of the stimuli, as listed in ds.sas{k}.typenames, is the same, 0 to compare independent of odrer; default is 0
%     - record_num_offset (int): value to add to the record number in displaying aarnings; default is 0
%
% Returns:
%   check (struct): summary of the consistency check, with fields
% 
%     - warnings (char): warning messages
%     - warn_bad (int): count of severe warnings that likely prevent processing
%     - nsets (int): number of records
%     - nstims_each (int 1-D array): nstims_each(k) is the number of stimuli in record k
%     - dim_list_each (cell array): dim_list_each{k} is the list of dimensions available in record k
%     - dim_list_union (int 1-D array): union of dim_list_each{:}
%     - dim_list_inter (int 1-D array): intersection of dim_list_each{:}
%     - typenames_each (cell array): typenames_each{k} is data_in.sas{k}.typenames, in original order
%     - typenames_union (cell array): union of typenames_each{:}, alphabetized
%     - typenames_inter (cell array): intersection of typenames_each{:}, alphabetized
%
%   opts_used (struct): opts, with defaults filled in
%
%  See also:  RS_KNIT_COORDSETS, RS_ALIGN_COORDSETS.
%
if (nargin<=1)
    opts=struct;
end
opts=filldefault(opts,'if_warn',0);
opts=filldefault(opts,'if_checkorder',1);
opts=filldefault(opts,'record_num_offset',0);
opts_used=opts;
check=struct;
check.warn_bad=0;
check.warnings=[];
%
nsets_ds=length(data_in.ds);
nsets_sas=length(data_in.sas);
nsets_sets=length(data_in.sets);
nsets=min([nsets_ds nsets_sas nsets_sets]);
%
nstims_each=zeros(1,nsets);
dim_list_each=cell(1,nsets);
dim_list_union=[];
typenames_each=cell(1,nsets);
typenames_union=[];
%check that ds, sas, sets is consistent
if nsets_sas~=nsets_ds
    wmsg=sprintf('number of records in metadata structure sas (%3.0f) and data structure ds (%3.0f) are inconsistent',nsets_sas,nsets_ds);
    check=rs_warning(wmsg,1,setfield(check,'if_warn',opts.if_warn));
end
if nsets_sets~=nsets
    wmsg=sprintf('number of records in metadata structure sets (%3.0f) and data structure ds (%3.0f) are inconsistent',nsets_sets,nsets_ds);
    check=rs_warning(wmsg,1,setfield(check,'if_warn',opts.if_warn));
end
for iset=1:nsets
    %check that number of stimuli is internally consistent
    nstims_each(iset)=data_in.sets{iset}.nstims;
    typenames_each{iset}=data_in.sas{iset}.typenames;
    if nstims_each(iset)~=data_in.sas{iset}.nstims
        wmsg=sprintf('number of stimuli in set %1.0f in sets (%3.0f) and sas (%3.0f) are inconsistent',iset+opts.record_num_offset,nstims_each(iset),data_in.sas{iset}.nstims);
        check=rs_warning(wmsg,1,setfield(check,'if_warn',opts.if_warn));
    end
    if nstims_each(iset)~=length(typenames_each{iset})
        wmsg=sprintf('number of stimuli in set %1.0f in sets (%3.0f) and typenames (%3.0f) are inconsistent',iset+opts.record_num_offset,nstims_each(iset),length(typenames_each{iset}));
        check=rs_warning(wmsg,1,setfield(check,'if_warn',opts.if_warn));
    end
    %dimension list must be unique and ascending and integer
    dim_list_each{iset}=data_in.sets{iset}.dim_list;
    if (any(dim_list_each{iset}~=round(dim_list_each{iset})) | (any(dim_list_each{iset}<=0)))
        wmsg=sprintf('dimension list record %3.0f must be positive integers',iset);
        check=rs_warning(wmsg,1,setfield(check,'if_warn',opts.if_warn));
    end
    dim_list_each{iset}=max(1,round(dim_list_each{iset}));
    if any(diff(dim_list_each{iset})<=0)
        wmsg=sprintf('dimension list for record %3.0f has non-unique values',iset);
        check=rs_warning(wmsg,1,setfield(check,'if_warn',opts.if_warn));
    end
    dim_list_each{iset}=unique(dim_list_each{iset});
    %
    if iset==1
        typenames_inter=typenames_each{iset};
        dim_list_inter=dim_list_each{iset};
    end
    typenames_union=union(typenames_union,typenames_each{iset});
    typenames_inter=intersect(typenames_inter,typenames_each{iset});
    %check that each array of coordinates is consistent with number of stimuli and dimension
    dim_remove=[];
    for idim_ptr=1:length(dim_list_each{iset})
        idim=dim_list_each{iset}(idim_ptr);
        if length(data_in.ds{iset})>=idim
            coords=data_in.ds{iset}{idim};
            if size(coords,1)~=nstims_each(iset)
                wmsg=sprintf('number of stimuli in coordinate array for record %3.0f and dimension %3.0f (%3.0f) is inconsistent with number of stimuli (%3.0f)',iset+opts.record_num_offset,idim,size(coords,1),nstims_each(iset));
                check=rs_warning(wmsg,1,setfield(check,'if_warn',opts.if_warn));
            end
            if size(coords,2)~=idim
                wmsg=sprintf('number of dimensions in coordinate array for record %3.0f and dimension %3.0f (%3.0f) is inconsistent with expected dimension (%3.0f)',iset+opts.record_num_offset,idim,size(coords,2),idim);
                check=rs_warning(wmsg,1,setfield(check,'if_warn',opts.if_warn));
            end
        else
            wmsg=sprintf('coordinate array for record %3.0f, dimension %3.0f is missing',iset+opts.record_num_offset,idim);
            check=rs_warning(wmsg,1,setfield(check,'if_warn',opts.if_warn));
            dim_remove=[dim_remove,idim];
        end
    end
    dim_list_each{iset}=setdiff(dim_list_each{iset},dim_remove);
    dim_list_union=union(dim_list_union(:)',dim_list_each{iset});
    dim_list_inter=intersect(dim_list_inter,dim_list_each{iset});
    %check if there are extra coordinate sets
    for idim=1:length(data_in.ds{iset})
        coords=data_in.ds{iset}{idim};
        if (size(coords,1)==nstims_each(iset) & size(coords,2)==idim)
            if ~ismember(idim,data_in.sets{iset}.dim_list)
                wmsg=sprintf('coordinate array for set %3.0f and dimension %3.0f is not listed in sets metadata; others may be processed',iset+opts.record_num_offset,idim);
                check=rs_warning(wmsg,0,setfield(check,'if_warn',opts.if_warn));
            end
        end
    end
end
%check that number of stimuli are consistent across datasets
if min(nstims_each)~=max(nstims_each)
    wmsg=sprintf('number of stimuli do not agree across datasets (min: %3.0f, max: %3.0f)',min(nstims_each),max(nstims_each));
    check=rs_warning(wmsg,1,setfield(check,'if_warn',opts.if_warn));
end
%check that stimulus names are consistent across datasets
if length(typenames_inter)~=length(typenames_union)
    wmsg=sprintf('stimulus names do not agree across datasets');
    check=rs_warning(wmsg,1,setfield(check,'if_warn',opts.if_warn));
else %only check order of typenames if they have same length and match in some order
    if opts.if_checkorder
        if_order=1;
        for iset=1:nsets-1
            if any(~strcmp(typenames_each{iset},typenames_each{iset+1}))
                if_order=0;
            end
        end
        if if_order==0
            wmsg=sprintf('stimulus orders do not agree across datasets');
            check=rs_warning(wmsg,1,setfield(check,'if_warn',opts.if_warn));
        end
    end
end %same stimulus names,independent of order?
if length(dim_list_union)~=length(dim_list_inter)
    wmsg=sprintf('dimension lists do not agree across datasets; intersection is available for processing'); %this is not severe, process the intersection
    if opts.if_warn
        check=rs_warning(wmsg,0,setfield(check,'if_warn',opts.if_warn));
        disp('discrepancies')
        disp(setdiff(dim_list_union,dim_list_inter));
    end
end
check.nsets=nsets;
check.nstims_each=nstims_each;
check.dim_list_each=dim_list_each;
check.dim_list_union=dim_list_union;
check.dim_list_inter=dim_list_inter;
check.typenames_each=typenames_each;
check.typenames_union=typenames_union;
check.typenames_inter=typenames_inter;
return
end

`rs_concat_coordsets(data_in1, data_in2, aux)`

[data_out,aux_out]=rs_concat_coordsets(data_in1,data_in2,aux) concatenates two dataset structures and checks the concatenated dataset structure for consistency

Input arguments:

data_in1 (struct) –
first dataset structure, containing \(n_1\) records, with fields
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record
data_in2 (struct) –

second dataset structure, containing \(n_2\) records, same format as data_in1
aux (struct) –
auxiliary options, may be omitted, with field
- opts_check (struct) – options for consistency checking, with field
- if_warn (int) – 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1

Output arguments:

data_out ( struct ) –

concatenated dataset structure with \(n_1 + n_2\) records, same format as as data_in1
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_check (struct) – aux.opts_check, with defaults filled in

Notes

aux may be omitted; defaults are filled in.
data_in1 and data_in2 are typically created by rs_get_coordsets, rs_align_coordsets, or rs_import_coordsets.
data_out.sets{k}.pipeline is copied from data_in1 or data_in2 and is not updated.

Source code in src/rs_concat_coordsets.m

function [data_out,aux_out]=rs_concat_coordsets(data_in1,data_in2,aux)
% [data_out,aux_out]=rs_concat_coordsets(data_in1,data_in2,aux)
% concatenates two `dataset structures` and checks the concatenated `dataset structure` for consistency
%
% Args:
%   data_in1 (struct): first `dataset structure`, containing $n_1$ records, with fields
%
%     - ds (cell array): `coordinate structure`, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
%     - sas (cell array): `stimulus metadata structure`, sas{k} is the stimulus metadata for the kth record
%     - sets (cell array): `set metadata structure`, sets{k} is the response metadata for the kth record
%
%   data_in2 (struct): second `dataset structure`, containing $n_2$ records, same format as `data_in1`
%
%   aux (struct): auxiliary options, may be omitted, with field
%
%     - opts_check (struct): options for consistency checking, with field
%
%       - if_warn (int): 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
% 
% Returns:
%   data_out (struct): concatenated `dataset structure` with $n_1 + n_2$ records, same format as  as `data_in1`
%
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_check (struct): aux.opts_check, with defaults filled in
%
% Note: Notes
%   -  aux may be omitted; defaults are filled in.
%   - `data_in1` and `data_in2` are typically created by `rs_get_coordsets`, `rs_align_coordsets`, or `rs_import_coordsets`.
%   -  data_out.sets{k}.pipeline is copied from data_in1 or data_in2 and is not updated.
%
% See also:
%   RS_IMPORT_COORDSETS, RS_AUX_CUSTOMIZE, RS_CHECK_COORDSETS, RS_EXTRACT_COORDSETS.
%
if (nargin<=2)
    aux=struct;
end
%set up sub-structure options
%
aux=filldefault(aux,'opts_check',struct); %options for other modules called
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
data_out=struct;
aux_out=struct;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
fns={'ds','sas','sets'}; %fields to concatenate
%
ns=zeros(2,length(fns)); %check that fields to concatenate have same length
for ifn=1:length(fns)
    fn=fns{ifn};
    ns(1,ifn)=length(data_in1.(fn));
    ns(2,ifn)=length(data_in2.(fn));
end
if any(min(ns,[],2)~=max(ns,[],2))
    wmsg='input structures have inconsistent lengths';
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
%
if aux_out.warn_bad==0
    n1=ns(1,1);
    n2=ns(2,1);
    for ifn=1:length(fns)
        fn=fns{ifn};
        data_out.(fn)=cell(1,n1+n2);
        for k=1:n1
            data_out.(fn){k}=data_in1.(fn){k};            
        end
        for k=1:n2
            data_out.(fn){n1+k}=data_in2.(fn){k};
        end
    end
    [check,opts_used]=rs_check_coordsets(data_out,aux.opts_check);
    aux_out.warnings=check.warnings;
    aux_out.warn_bad=check.warn_bad;
else
    disp('cannot proceed');
    disp(aux_out.warnings);
end
%    
aux_out.opts_check=aux.opts_check;
return
end

`rs_dirfit_choicedata(choices, aux)`

[dirfit,aux_out]=rs_dirfit_choicedata(choices,aux) finds the maximum-likelihood fit of a symmetric Dirichlet distribution to choice probabilities, optionally including a discrete component at p=0.5

Input arguments:

choices (int 2-D array) –

choices(:,1) is the number of times the first difference was judged more similar than the second difference; choices(:,2) is the number of times the comparison was made; this is typically the last two columns of data_comp as returned by rs_read_choicedata
aux (struct) –
a structure, can be omitted, with fields
- opts_dirfit (struct) – options for fitting Dirichlet parameters, can be omitted, with fields
  - if_log (int) – 1 to log progress, 0 to omit; default is 1
  - if_fit_a (int) – 1 to fit the shape parameter ('a') with an assumed value of the discrete parameter, 0 omits, default is 1
  - fixed_h (float) – value of 'h' to use for fitting just 'a', default is 0, which corresponds to zero weight for the discrete part
  - if_fit_h (int) – 1 to fit the discrete parameter ('h') with an assumed value of the shape parameter 'a', 0 omits, default is 0
  - fixed_a (float) – value of 'a' to use for fitting 'h', default is 1, which corresponds to a flat distribution
  - if_fit_ah (int) – 1 to fit the shape parameter 'a' and the discrete parameter 'h' simultaneously, 0 omits, default is 0; see note below
- opts_check (struct) – options for consistency checking, with field
  - if_warn (int) – 1 to show warnings, 0 to suppress; default is 1

Output arguments:

dirfit ( struct ) –
the fitted Dirichlet parameter values, a structure with fields
- nchoices (int) – number of choices used in the analysis, i.e., the number of rows of choices that has a nonzero element in the final column
- a (struct) – analysis of the Dirichlet shape parameter 'a' (present only if if_fit_a=1), with fields
  - val (float) – maximum-likelihood value for a
  - llnat_per_choice (float) – log likelihood (natural log) per choice probability
  - llnat_per_trial (float) – log likelihood (natural log) per trial
  - exitflag (int) – exit flag from fminbnd optimization
  - output (struct) – detailed output from fminbnd optimization
  - optimset (struct) – optimization options used in fminbnd optimization
  - jack_val (float 1-D array) – maximum-likelihood value for 'a' for each jackknife, only present if if_stats=1
  - jack_llnat_per_choice (float 1-D array) – log likelihood (natural log) per choice probability for each jackknife, only present if if_stats=1
  - jack_sem (float) – jackknife standard error of measurement for a, only present if if_stats=1
  - fixed_h (float) – value of 'h' used for fitting just 'a'
- h (struct) – analysis of the Dirichlet shape parameter 'h' (present only if if_fit_h=1), with fields
  - val (float) – maximum-likelihood value for a
  - llnat_per_choice (float) – log likelihood (natural log) per choice probability
  - llnat_per_trial (float) – log likelihood (natural log) per trial
  - exitflag (int) – exit flag from fminbnd optimization
  - output (struct) – detailed output from fminbnd optimization
  - optimset (struct) – optimization options used in fminbnd optimization
  - jack_val (float 1-D array) – maximum-likelihood value for 'h' for each jackknife, only present if if_stats=1
  - jack_llnat_per_choice (float 1-D array) – log likelihood (natural log) per choice probability for each jackknife, only present if if_stats=1
  - jack_sem (float) – jackknife standard error of measurement for 'h', only present if if_stats=1
  - fixed_a (float) – value of 'a' used for fitting just 'h'
- ah (struct) – joint analysis of the shape parameter and discrete parameter (present only if if_fit_ah=1), with fields
  - val (float 1-D array) – val(1) is maximum-likelihood value for a, val(2) is maximum-likelihood value for h
  - ll_per_choice (float) – log likelihood per choice probability
  - llnat_per_trial (float) – log likelihood (natural log) per trial
  - exitflag (int) – exit flag from `fminsearch optimization
  - output (struct) – detailed output from fminsearch optimization
  - optimset (struct) – optimization options used in fminsearch optimization
  - jack_val (float 2-D array) – maximum-likelihood value for a (jack_val(1,:)) and h (jack_val(2,:) for each jackknife, only present if if_stats=1
  - jack_llnat_per_choice (float 1-D array) – log likelihood (natural log) per choice probability for each jackknife, only present if if_stats=1
  - jack_sem (float 1-D array) – jack_sem(1) is jackknife standard error of measurement for a, jack_sem(2) is jackknife standard error of measurement for h, only present if if_stats=1
- jack_select (int 1-D array) – list of choices omitted on each jackknife (empty if if_stats=0)
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_dirfit (struct) – aux,opts_dirfit with defaults and values used
- opts_check (struct) – aux.opts_check, with defaults filled in

Notes re fitting, log likelihoods and statistics

The distribution of observed probabiliies p is fitted to a symmetric Dirichlet distribution with parameter 'a', i.e., P(p)=(p^(a-1))((1-p)^(a-1))/B(a-1,a-1), where B is the beta function
- Each nonzero row of 'choices' is considered an independent observation of the chioce probability distribution; rows containing zeros are ignored
- If if_fit_ah=1, it is also fitted to a symmetric Dirichlet distribution mixed with a discrete component at p=0.5, with weights 1-h and h, respectively
Log likelihoods use natural logs, and are normalized by the number of nonzero rows of 'choices'
Minimal number of choice probabilities (nonzero rows of 'choices') needed
- At least 2 choice probabilities are needed to fit the symmetric Dirichlet distribution
- At least 3 choice probabilities are needed to fit the distribution with an additional discrete component
- For a jacknife estimate of the s.e.m., at least one additional choice probability is needed
- If all empirical choice probabilities are 0 or 1, the estimated parameter values will go to the limits allowed for the fitted parameters, 'a_limits' and 'h_limits'
- This is a bare minimum for non-degeneracy. Useful fits typically require many more choice probabilities.
Optimization behavior can be controlled by aux.opts_dirfit.a_optimset and aux.opts_dirfit.ah_optimset. For example, to displahy each iteration for fitting 'a', set aux.opts_dirfit.a_optimset.Display='iter';

See also: rs_read_choicedata, loglik_beta_discrete, FMINSEARCH, FMINBND.

Source code in src/rs_dirfit_choicedata.m

function [dirfit,aux_out]=rs_dirfit_choicedata(choices,aux)
% [dirfit,aux_out]=rs_dirfit_choicedata(choices,aux) finds the maximum-likelihood fit of a symmetric Dirichlet distribution to choice probabilities, optionally including a discrete component at p=0.5
% 
% Args:
%   choices (int 2-D array): choices(:,1) is the number of times the first difference was judged __more similar than__ the second difference; choices(:,2) is the number of times the comparison was made; this is typically the last two columns of data_comp as returned by `rs_read_choicedata`
%
%   aux (struct): a structure, can be omitted, with fields 
%
%     - opts_dirfit (struct): options for fitting Dirichlet parameters, can be omitted, with fields
%
%         - if_log (int): 1 to log progress, 0 to omit; default is 1
%         - if_fit_a (int): 1 to fit the shape parameter ('a') with an assumed value of the discrete parameter, 0 omits, default is 1
%         - fixed_h (float): value of 'h' to use for fitting just 'a', default is 0, which corresponds to zero weight for the discrete part
%         - if_fit_h (int): 1 to fit the discrete parameter ('h') with an assumed value of the shape parameter 'a', 0 omits, default is 0
%         - fixed_a (float): value of 'a' to use for fitting 'h', default is 1, which corresponds to a flat distribution
%         - if_fit_ah (int): 1 to fit the shape parameter 'a' and the discrete parameter 'h' simultaneously, 0 omits, default is 0; see note below
%
%         - **Options for statistics and shuffles**
%         - if_stats (int): 1 to compute jackknife standard error of measurement, 0 does not; default is 0
%         - if_frozen (int): random number control; 1 for same numbers every run, 0 for different random numbers each run, negative integer for a fixed seed each run, default is 1
%         - njacks_max (int): maximum number of choice probabilities to remove for jackknifing (ignored if if_stats=0), default is 500
%
%         - **Options to control optimization details**
%         - a_limits (float): allowed range for 'a' (shape), when fitting only 'a', default is [10^-2 10^2]
%         - h_limits (float): allowed range for 'h' (discrete weight) when fitting only 'h', default is [0 1]
%         - a_optimset (struct): non-default optimizations parameters for fitting 'a', with `fminbnd`, default is struct()
%         - h_optimset (struct): non-default optimizations parameters for fitting 'h', with `fminbnd`, default is struct()
%         - ah_optimset (struct): non-default optimizations parameters for fitting 'a' and 'h', with `fminsearch`, default is struct()
% 
%     - opts_check (struct): options for consistency checking, with field
%
%         - if_warn (int): 1 to show warnings, 0 to suppress; default is 1
%
% Returns:
%   dirfit (struct): the fitted Dirichlet parameter values, a structure with fields
%
%      - nchoices (int): number of choices used in the analysis, i.e., the number of rows of choices that has a nonzero element in the final column 
%      - a (struct): analysis of the Dirichlet shape parameter 'a' (present only if if_fit_a=1), with fields
%
%          - val (float): maximum-likelihood value for a
%          - llnat_per_choice (float): log likelihood (natural log) per choice probability
%          - llnat_per_trial (float): log likelihood (natural log) per trial
%          - exitflag (int): exit flag from `fminbnd` optimization
%          - output (struct): detailed output from `fminbnd` optimization
%          - optimset (struct): optimization options used in `fminbnd` optimization
%          - jack_val (float 1-D array): maximum-likelihood value for 'a' for each jackknife, only present if if_stats=1
%          - jack_llnat_per_choice (float 1-D array): log likelihood (natural log) per choice probability for each jackknife, only present if if_stats=1
%          - jack_sem (float): jackknife standard error of measurement for a, only present if if_stats=1
%          - fixed_h (float): value of 'h' used for fitting just 'a'
%
%      - h (struct): analysis of the Dirichlet shape parameter 'h' (present only if if_fit_h=1), with fields
%
%          - val (float): maximum-likelihood value for a
%          - llnat_per_choice (float): log likelihood (natural log) per choice probability
%          - llnat_per_trial (float): log likelihood (natural log) per trial
%          - exitflag (int): exit flag from `fminbnd` optimization
%          - output (struct): detailed output from `fminbnd` optimization
%          - optimset (struct): optimization options used in `fminbnd` optimization
%          - jack_val (float 1-D array): maximum-likelihood value for 'h' for each jackknife, only present if if_stats=1
%          - jack_llnat_per_choice (float 1-D array): log likelihood (natural log) per choice probability for each jackknife, only present if if_stats=1
%          - jack_sem (float): jackknife standard error of measurement for 'h', only present if if_stats=1
%          - fixed_a (float):  value of 'a' used for fitting just 'h'
%
%      - ah (struct): joint analysis of the shape parameter and discrete parameter (present only if if_fit_ah=1), with fields
%
%          - val (float 1-D array): val(1) is maximum-likelihood value for a, val(2) is maximum-likelihood value for h
%          - ll_per_choice (float): log likelihood per choice probability
%          - llnat_per_trial (float): log likelihood (natural log) per trial
%          - exitflag (int): exit flag from `fminsearch optimization
%          - output (struct): detailed output from `fminsearch` optimization
%          - optimset (struct): optimization options used in `fminsearch` optimization
%          - jack_val (float 2-D array): maximum-likelihood value for a (jack_val(1,:)) and h (jack_val(2,:) for each jackknife, only present if if_stats=1
%          - jack_llnat_per_choice (float 1-D array): log likelihood (natural log) per choice probability for each jackknife, only present if if_stats=1
%          - jack_sem (float 1-D array): jack_sem(1) is jackknife standard error of measurement for a, jack_sem(2) is jackknife standard error of measurement for h, only present if if_stats=1
%
%      - jack_select (int 1-D array): list of choices omitted on each jackknife (empty if if_stats=0)
% 
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_dirfit (struct): aux,opts_dirfit with defaults and values used
%     - opts_check (struct): aux.opts_check, with defaults filled in
%
% Note: Notes re fitting, log likelihoods and statistics
%     - The distribution of observed probabiliies p is fitted to a symmetric Dirichlet distribution with parameter 'a', i.e., P(p)=(p^(a-1))((1-p)^(a-1))/B(a-1,a-1), where B is the beta function
%
%         - Each nonzero row of 'choices' is considered an independent observation of the chioce probability distribution; rows containing zeros are ignored
%         - If if_fit_ah=1, it is also fitted to a symmetric Dirichlet distribution mixed with a discrete component at p=0.5, with weights 1-h and h, respectively
%
%     - Log likelihoods use natural logs, and are normalized by the number of nonzero rows of 'choices'
%     - Minimal number of choice probabilities (nonzero rows of 'choices') needed
%
%         - At least 2 choice probabilities are needed to fit the symmetric Dirichlet distribution
%         - At least 3 choice probabilities are needed to fit the distribution with an additional discrete component
%         - For a jacknife estimate of the s.e.m., at least one additional choice probability is needed
%         - If all empirical choice probabilities are 0 or 1, the estimated parameter values will go to the limits allowed for the fitted parameters, 'a\_limits' and 'h\_limits'
%         - This is a bare minimum for non-degeneracy.  Useful fits typically require many more choice probabilities. 
%
%     - Optimization behavior can be controlled by aux.opts_dirfit.a_optimset and aux.opts_dirfit.ah_optimset.  For example, to displahy each iteration for fitting 'a', set aux.opts_dirfit.a_optimset.Display='iter';
%
% See also: RS_READ_CHOICEDATA, LOGLIK_BETA_DISCRETE, FMINSEARCH, FMINBND.
%
if (nargin<=1)
    aux=struct;
end
aux=filldefault(aux,'opts_dirfit',struct);
aux.opts_dirfit=filldefault(aux.opts_dirfit,'if_fit_a',1);
aux.opts_dirfit=filldefault(aux.opts_dirfit,'if_fit_h',0);
aux.opts_dirfit=filldefault(aux.opts_dirfit,'if_fit_ah',0);
aux.opts_dirfit=filldefault(aux.opts_dirfit,'fixed_a',1);
aux.opts_dirfit=filldefault(aux.opts_dirfit,'fixed_h',0);
aux.opts_dirfit=filldefault(aux.opts_dirfit,'if_log',1);
aux.opts_dirfit=filldefault(aux.opts_dirfit,'a_limits',[10^-2 10^2]);
aux.opts_dirfit=filldefault(aux.opts_dirfit,'h_limits',[0 1]);
aux.opts_dirfit=filldefault(aux.opts_dirfit,'if_frozen',1);
aux.opts_dirfit=filldefault(aux.opts_dirfit,'if_stats',0);
aux.opts_dirfit=filldefault(aux.opts_dirfit,'njacks_max',500);
aux.opts_dirfit=filldefault(aux.opts_dirfit,'a_optimset',struct());
aux.opts_dirfit=filldefault(aux.opts_dirfit,'h_optimset',struct());
aux.opts_dirfit=filldefault(aux.opts_dirfit,'ah_optimset',struct());
%
aux=filldefault(aux,'opts_check',struct);
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
aux=rs_aux_customize(aux,'rs_dirfit_choicedata');
%
aux_out=struct;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
midpoint_prob=0.5; 
%
%set up random number generator
%
if_frozen=aux.opts_dirfit.if_frozen;
if (if_frozen~=0) 
    rng('default');
    if (if_frozen<0)
        rand(1,abs(if_frozen));
    end
else
    rng('shuffle');
end
%
choices_nz=find(choices(:,2)>0);
choices_used=choices(choices_nz,:);
probs=choices_used(:,1)./choices_used(:,2);
nchoices=length(choices_nz);
ntrials=sum(choices(:,2));
%
if aux.opts_dirfit.if_log
    pstring=[];
    if aux.opts_dirfit.if_fit_a
        pstring=cat(2,pstring,sprintf('a (h=%5.3f); ',aux.opts_dirfit.fixed_h'));
    end
    if aux.opts_dirfit.if_fit_h
        pstring=cat(2,pstring,sprintf('h (a=%5.3f); ',aux.opts_dirfit.fixed_a'));
    end
    if aux.opts_dirfit.if_fit_ah
        pstring=cat(2,pstring,sprintf('(a,h); '));
    end
    if length(pstring)>0
        pstring=pstring(1:end-2);
    end
    disp(sprintf('fitting Dirichlet params (choices used: %5.0f; trials used: %5.0f; probability range: [%8.6f %8.6f]): %s',...
        nchoices,ntrials,min(probs),max(probs),pstring));
end
%
dirfit=struct;
dirfit.nchoices=nchoices;
if any(choices(:,1)>choices(:,2))
    wmsg=sprintf('some choice counts are greater than the number of trials');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
%
% enough data? at least two choices needed to fit a, three choices for a and h, and one extra if doing jackknifes
%
choices_needed=2+aux.opts_dirfit.if_fit_ah; %minimal choices needed
if nchoices<choices_needed
    wmsg=sprintf('insufficient choices available for fitting; at least %2.0f needed',choices_needed);
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
else
    if nchoices<choices_needed+aux.opts_dirfit.if_stats
        wmsg=sprintf('insufficient choices available for computing statistics; at least %2.0f needed',choices_needed+aux.opts_dirfit.if_stats);
        aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
        aux.opts_dirfit.if_stats=0; %remove stats
    end
end
if all(or(probs==0,probs==1))
   wmsg=sprintf('all probabilities are 0 or 1; fitted parameter values are unreliable');
   aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
%
aux_out.opts_dirfit=aux.opts_dirfit;
%
if aux_out.warn_bad>0
    disp('cannot proceed');
    disp(aux_out.warnings);
    return
end
if aux.opts_dirfit.if_stats
    njacks=min(aux.opts_dirfit.njacks_max,nchoices);
    if njacks<nchoices
        jack_select=sort(randperm(nchoices,njacks));
    else
        jack_select=[1:nchoices];
    end
    if aux.opts_dirfit.if_fit_a
        dirfit.a.jack_val=zeros(1,njacks);
        dirfit.a.jack_nllnat_per_choice=zeros(1,njacks);
    end
    if aux.opts_dirfit.if_fit_h
        dirfit.h.jack_val=zeros(1,njacks);
        dirfit.h.jack_nllnat_per_choice=zeros(1,njacks);
    end
    if aux.opts_dirfit.if_fit_ah
        dirfit.ah.jack_val=zeros(2,njacks);
        dirfit.ah.jack_nllnat_per_choice=zeros(1,njacks);
    end
else
    njacks=0;
    jack_select=[];
end
dirfit.jack_select=jack_select;
%
a_opts=optimset(optimset('fminbnd'),aux.opts_dirfit.a_optimset);
h_opts=optimset(optimset('fminbnd'),aux.opts_dirfit.h_optimset);
%quantities needed for discrete part
ah_opts=optimset(optimset('fminsearch'),aux.opts_dirfit.ah_optimset);
a_init=1;
h_init=0;
opts_beta=struct;
opts_beta.qvec=midpoint_prob;
%
for ijack=0:njacks
    if (ijack==0)
        c=choices_used;
    else
        c=choices_used(setdiff(1:nchoices,jack_select(ijack)),:);
    end
    %
    %fit a (without discrete part)
    %
    if aux.opts_dirfit.if_fit_a
        [a_fit,a_fit_nll,a_fit_exitflag,a_fit_output]=fminbnd(@(x) -loglik_beta_discrete(x,c,setfield(opts_beta,'hvec',aux.opts_dirfit.fixed_h)),aux.opts_dirfit.a_limits(1),aux.opts_dirfit.a_limits(2),a_opts);
        if (ijack==0)
            dirfit.a.val=a_fit;
            dirfit.a.llnat_per_choice=-a_fit_nll/nchoices;
            dirfit.a.llnat_per_trial=-a_fit_nll/ntrials;
            dirfit.a.exitflag=a_fit_exitflag;
            dirfit.a.output=a_fit_output;
            dirfit.a.optimset=a_opts;
            dirfit.a.fixed_h=aux.opts_dirfit.fixed_h;
            a_init=a_fit; %for use in fitting (a,h)
        else
            dirfit.a.jack_val(1,ijack)=a_fit;
            dirfit.a.jack_nllnat_per_choice(1,ijack)=-a_fit_nll/nchoices;
        end
    end %if_fit_a
    %
    %fit h 
    %
    if aux.opts_dirfit.if_fit_h
        [h_fit,h_fit_nll,h_fit_exitflag,h_fit_output]=fminbnd(@(x) -loglik_beta_discrete(aux.opts_dirfit.fixed_a,c,setfield(opts_beta,'hvec',x)),aux.opts_dirfit.h_limits(1),aux.opts_dirfit.h_limits(2),h_opts);
        if (ijack==0)
            dirfit.h.val=h_fit;
            dirfit.h.llnat_per_choice=-h_fit_nll/nchoices;
            dirfit.h.llnat_per_trial=-h_fit_nll/ntrials;
            dirfit.h.exitflag=h_fit_exitflag;
            dirfit.h.output=h_fit_output;
            dirfit.h.optimset=h_opts;
            dirfit.h.fixed_a=aux.opts_dirfit.fixed_a;
            h_init=h_fit; % for use in fitting (a,h)
        else
            dirfit.h.jack_val(1,ijack)=h_fit;
            dirfit.h.jack_nllnat_per_choice(1,ijack)=-h_fit_nll/nchoices;
        end
    end %if_fit_h
    %
    %fit both
    %
    if aux.opts_dirfit.if_fit_ah
        if (ijack==0)
            ah_init=[a_init;h_init];
        else
            ah_init=dirfit.ah.val; %use starting point from full dataset
        end
        [ah_fit,ah_fit_nll,ah_fit_exitflag,ah_fit_output]=fminsearch(@(x) -loglik_beta_discrete(x(1),c,setfield(opts_beta,'hvec',x(2))),ah_init,ah_opts);
        if (ijack==0)
            dirfit.ah.val=ah_fit;
            dirfit.ah.llnat_per_choice=-ah_fit_nll/nchoices;
            dirfit.ah.llnat_per_trial=-ah_fit_nll/ntrials;
            dirfit.ah.exitflag=ah_fit_exitflag;
            dirfit.ah.output=ah_fit_output;
            dirfit.ah.optimset=ah_opts;
        else
            dirfit.ah.jack_val(:,ijack)=ah_fit;
            dirfit.ah.jack_nllnat_per_choice(1,ijack)=-ah_fit_nll/nchoices;
        end
    end %if_fit_ah
end %ijack
%
if aux.opts_dirfit.if_stats
    if aux.opts_dirfit.if_fit_a
        dirfit.a.jack_sem=rs_dirfit_jack(dirfit.a.jack_val,nchoices);
    end
    if aux.opts_dirfit.if_fit_h
        dirfit.h.jack_sem=rs_dirfit_jack(dirfit.h.jack_val,nchoices);
    end
    if aux.opts_dirfit.if_fit_ah
        dirfit.ah.jack_sem=rs_dirfit_jack(dirfit.ah.jack_val,nchoices);
    end
end
%
return
end

`rs_disp_coordsets(data_in, aux)`

aux_out=rs_disp_coordsets(data_in,aux) displays one or more views of pairs or triplets of coordinates in a dataset structure

Multiple views can be plotted, one eachin a subplot of the same figure. This is particularly helpful if the dimensionality of the coordinates is high: each subplot could show a different combination of two or three coordinates. Plots of two coordinates will produce a planar view; plots of three coordinates will produce an isometric 3D view. Subplots are left in the 'hold on' state.

Input arguments:

data_in (struct) –
dataset structure to be processed, with fields
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record
aux (struct) –
auxiliary inputs, may be omitted, with fields
- opts_disp (struct) – options for display, with fields to contol
- if_warn (int) – 1 to display warnings related to plot configurations, 0 to suppress; default is 1
- opts_check (struct) – options for consistency checking, with field
  - if_warn (int) – 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1

Output arguments:

aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_disp (struct) – aux.opts_disp, with defaults and overrides filled in, including fig_handle (handle to the figure) and axis_handles (handles to the subplot axes)

Note regarding customization

The default figure position can be changed by editing the line containing generic.opts_disp.fig_position in rs_aux_defaults_define, running it once, and saving the workspace as rs_aux_defaults.mat.
The default font size for axis labels can be changed by editing the line containing generic.opts_disp.axis_font_size in rs_aux_defaults_define, running it once, and saving the workspace as rs_aux_defaults.mat.
The default prefix for the axis label can be changed by editing the line containing generic.opts_disp.axis_label_prefix in rs_aux_defaults_define, running it once, and saving the workspace as rs_aux_defaults.mat.

Note regarding plot formatting

By default, each record is plotted with a different color, a solid dot marker, and no connecting lines.
These choices can be changed by 'set_colors', 'set_markers', 'set_markersizes', 'set_filled', 'set_colors_filled', 'set_alphas' (but alpha blending may not be availble on all systems).
The above specifiers can be singletons or cell arrays, and are indexed by the record position in data_in. If there are more records than specifiers, the specifiers are cycled.
The data points of each record are, by default, not interconnected. This can be changed with the 'connect_data*' options.
The data points between records are, by default, not interconnected . This can be changed with the 'connect_sets*' options.

Source code in src/rs_disp_coordsets.m

function aux_out=rs_disp_coordsets(data_in,aux)
% aux_out=rs_disp_coordsets(data_in,aux) displays one or more views of pairs or triplets of coordinates in a `dataset structure`
%
% Multiple views can be plotted, one eachin a subplot of the same figure. This is particularly helpful if the dimensionality
% of the coordinates is high:  each subplot could show a different combination of two or three coordinates.
% Plots of two coordinates will produce a planar view; plots of three coordinates will produce an isometric 3D view. Subplots are left in the 'hold on' state.
%
% Args:
%   data_in (struct): `dataset structure` to be processed, with fields
%
%     - ds (cell array): `coordinate structure`, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
%     - sas (cell array): `stimulus metadata structure`, sas{k} is the stimulus metadata for the kth record
%     - sets (cell array): `set metadata structure`, sets{k} is the response metadata for the kth record
%
%   aux (struct): auxiliary inputs, may be omitted, with fields
%
%     - opts_disp (struct): options for display, with fields to contol
%
%         - **Data selection**
%         - set_select (int 1-D array): list of records to show; defaults is [1:length(data_in.ds)]
%         - dim_select (int): dimension to show (>=2); i.e., dim_select=k results in display of the coordinates in data_in.ds{set_select}{k}; default is 3 unless only two dimensions are available; must be at least 2
%         - coord_group_size (int): number of coordinates to display together, in range [2 3]; default is min(dim_select,number of dimensions available,3)
%         - coord_group_method (char): method of selecting coordinates
%
%             - 'all': (default) plot all combinations
%             - 'keeplow': keep 1 to dim_select-1 and step the highest; [dim_select,coord_group_size]=[5,3] yields [1 2 3],[1 2 4],[1 2 5]
%             - 'keepone': keep one dimension and step the rest;        [dim_select,coord_group_size]=[5,3] yields [1 2 3],[1 2 4],[1 2 5],[1 3 4],[1 3 5],[1 4 5]
%             - 'rolling': rolling contiguous subsets;                  [dim_select,coord_group_size]=[5,3] yields [1 2 3],[2 3 4],[3 4 5],[4 5 1],[5 1 2]
%             - 'onlylowest': only the lowest dimensions;               [dim_select,coord_group_size]=[5,3] yields [1 2 3]
%             - 'list': specify a list in opts_disp.coord_groups, as an array with coord_group_size columns, e.g., opts_disp.coord_groups=[1 2 3;1 4 5;1 6 7] creates three subplots, with coordinates {1,2,3} in the first, {1,4,5} in the second, {1,6,7} in the third
%
%         - coord_groups (int 2-D array): groups of coordinates to show together if coord_group_method='list', as rows (of length coord_group_size) of an integer array; each row will generate one subplot
%         - data_show_method (char): which data points to show, options are 'all', 'none', 'first', 'last', 'list'; default is 'all'
%         - data_show_list (int 1-D array): list of data points to show, if data_show_method='list';  points plotted are data_in.ds{set_select}{k}(data_show_list,:);  This applies to data in all records. To disable display of individual data points in specific records, set their coordinates to NaN.
%
%         - **Figure and subplot control**
%         - fig_handle (handle): handle to figure; will be created if empty or not provided
%         - fig_position (int 1-D array): position parameters [left bottom width height] for figure to be created; see note below regarding customization
%         - fig_name (char): title for figure; default is dimension shown
%         - axis_handles (cell array of handles): handle to axes, one for each subplot, will be created empty or not provided
%
%         - **Labels**
%         - set_labels (char or cell array of char): labels for each record that will appear in legend; defaults is 'set 1', etc.; cannot be empty; see note regarding plot formatting
%         - data_label_setsel_method (char): selects which records to label individual points, options are 'all','none', 'first' , 'last', or 'list'; default is 'first'; note that 'all','first', and 'last' apply to the records shown
%         - data_label_setsel_list (int 1-D array): list of datasets to label, if data_label_setsel_method='list'
%         - data_label_method (char): selects which data points to label, options are 'all', 'none', 'first', 'last', 'list'; default is 'all'; labels are taken from data_in.sas{k}.typenames; note that 'all','first',and 'last' refer to data points shown
%         - data_label_list (int 1-D array): list of data points to label, if data_label_method='list'
%         - data_label_font_size (int): font size for data labels, default is axis_font_size
%         - data_label_interpreter (char): interpreter for labeling data, [] (default) uses system default, alternatively 'none','tex','latex'
%         - data_label_typenames_vary (int): 1 to allow typenames to vary across datasets, 0 does not; default is 0
%         - callout_amount (float): moves the position of a label away from the data point, specified in units of rms deviation of data from callout_center; default is 0
%         - callout_colors (cell array of color specifiers): color for callout lines connecting labels and points; default is {'k'}; can also be 'set_colors' to match set_colors
%         - callout_linestyles (cell array of char): line styles for above callout lines; default is {'-.'}
%         - callout_linewidths (int 1-D array): line widths for above callout lines; default is 1
%         - callout_center (char or int 1-D array): callout lines point from the specified location outward; default is 'centroid' (centroid of data), can also be 'zero' or a coordinate value, of length 'dim_select' 
%
%         - **Formatting: axis and views**
%         - axis_font_size (int): font size for axis; default is 8; see note below re customization
%         - axis_label_font_size (char): font size for axis labels; default is axis_font_size
%         - axis_label_prefix (char): prefix for axis label, default is 'coord'; see note below regarding customization
%         - axis_labels (cell array of char): cell array of strings, cycled through if necessary, with text for axis labels.  If empty, then axis labels are genrated from axis_label_prefix
%         - axis_view (int or float 1-D array or cell array): 3-D view descriptor, default is 3 (standard 3-d view), 2 is 2-d view; can also be azimuth-elevation pair; standard 3-d view is [-37.5 30]; can be also be cell array of view specifiers, is cycled through for each subplot
%         - axis_equal (int): 1 to set axes to have equal scales, 0 autoscales; default is 1
%         - axis_range (char): 'tight' to set axis range to limits of data, 'auto' for autoscaling, or 'list' to specify by axis_range_list; default is 'tight'
%         - axis_range_list (float 2-D array): axis range specification, as rows of [low, high] values, one for each coordinate plotted; cycled through by rows if necessary
%
%         - **Formatting: points and lines for each record**
%         - set_colors (color specifier or cell array of color specifiers): color assigned to each record; default is {'k','b','c','m','r',[0.5 0.5 0],'g'}; elements can be any valid color specifier; see note regarding plot formatting
%         - set_markers (char or cell array of char): marker for each record; defaults is {'.'}; see note below regarding plot formatting
%         - set_markersizes (int 1-D array): marker size for each record; default is 8; see below note regarding plot formatting
%         - set_filled (int 1-D array): 1 for records in which markers are filled, 0 for unfilled if possible; default is 0; note that only some marker, e.g., o,s,h,p can be unfilled
%         - set_colors_filled (color specifer or cell array of color specifiers): color for inside of filled symbols for each record, ignored if set_filled=0; default is set_colors
%         - set_alphas (float 1-D array): alpha blending for each record, default is 1 (opaque); note that alpha-blending (transparency) may not be availble on all systems
%         - connect_data_method (char): which pairs of data points to connect within a record; default is 'none'; options refer to the points selected for display by 'data_show_method'; see note below regarding plot formatting
%
%             - 'none': no connections between points
%             - 'all': connect all pairs of points
%             - 'star' or 'star_first': connect all points to first
%             - 'star_last': connect all points to last
%             - 'chain': connect first point =to second, second point to third, ..., and next-to-last point to last point
%             - 'circuit': same as 'chain' but adds a connection from last point to first
%             - 'list': specify the pairs of points in connect_data_list
% 
%         - connect_data_list (int 2-D array): two-column array of data points to connect (if connect_data_method='list')
%         - connect_data_linestyles (char or cell array of char): line styles assigned to connections within each record, default is 'none' (disconnected)
%         - connect_data_linewidths (int 1-D array): line widths for connections within each record; default is 1
%
%         - **Formatting: connections between records**
%         - connect_sets_method (char): which records to connect; default is 'none'; options refer to the records selected for display by 'set_select'; see note below regarding plot formatting
%
%             - 'none': no connections between data from different records
%             - 'all': connect all pairs of records
%             - 'star' or 'star_first': connect all records to first
%             - 'star_last': connect all records to last
%             - 'chain': connect first record to second, second record to third, ..., and next-to-last record to last record
%             - 'circuit': same as 'chain' but adds a connection from last record to first
%             - 'list': specify the pairs of records in connect_sets_list
% 
%         - connect_sets_list(int 2-D array): two-column array of records to connect, used if connect_sets_method='list'
%         - connect_sets_data_method (char): selects the data points to connect between records: any of the options for data_label_method or 'labeled', which connects all data points designated by 'data_label_method'; default is 'all'
%         - connect_sets_data_list (int 1-D array): data points to connect between records, if connect_sets_data_method='list'
%         - connect_sets_color_mode (char): specifies how the connecting semgent between records is colored; default is 'split'
%
%             - 'first': use color of first record in connected pair
%             - 'last': use color of last record in connected pair
%             - 'split': first half of segment matches first record in connected pair; second half of segmentmatches second record in connected pair
%             - 'list': specify colors in connect_sets_colors
%
%         - connect_sets_colors (color specifier or cell array of color specifiers): if connect_sets_color_mode='list';, these are the colors for for connecting segments; cycled through if necessary
%         - connect_sets_linestyles (char or cell array of char): line styles assigned to connections between records; default is  '-' (solid); cycled through if necessary
%         - connect_sets_linewidths (int 1-D array): line widths assigned to connections between records; default is 1; cycled through if necessary
%
%         - **Formatting: positioning** 
%         - set_offsets (float 2-D array or char): this allows the data in each record to be offset to avoid overlap; default is no offset; specified by 0
%
%              - if an array, each row, of length dim_select (which wll be truncated or padded as needed) specifies the offset for the corresponding record in `data_in`
%              - 'margin_amount' puts a margin of set_offsets_margin_amount between each dataset and the next
%              - 'margin_fraction' puts a fractional margin of set_offsets_margin_fraction * average span of adjacent sets
%
%         - set_offsets_margin_amount (float 1-D array): absolute margin between datasets if set_offsets='margin_amount; defaults to ones(1,dim_select); can be 0 or negative, truncated or padded to dim_select
%         - set_offsets_margin_fraction (float 1-D array): fractional margin between datasets if set_offsets='margin_fraction; defaults to zeros(1,dim_select); can be 0 or negative, truncated or padded to dim_select
%         - set_offsets_coordchoices (int or char or cell array of char): if set_offsets='margin_amount' or 'margin_fraction', this specifies which coordinate is offset; can be 'first','last','all', or a subset of [1:dim_select]; can also be a cell array of subsets
%
%         - **Formatting: perpendiculars from points to walls**
%         - perp_data_dims (int or int 1-D array): 1 to drop perpendiculars along that dimension,-1 for perpendicular to top, 0 to omit; default is 0; cycled through if necessary
%         - perp_data_linestyles (char or cell array of char): line styles for perpendiculars for each record, or 'use_data' to use the style specified by 'connect_data_linestyles' for connecting data points; default is '-' (solid); cycled through if necessary
%         - perp_data_linewidths (int 1-D array): line widths for perpendiculars for each record; default is 1
%         - perp_data_usemarkers (int): 1 to use markers from original data at foot of perpendicular, 0 for no markers; default is 1
%
%         - **Formatting: legend, box, grid**
%
%         - if_legend (int): 1 to include legend, 0 to omit, -1 to omit from all subplots but to create an extra subplot that only has the legend; default is 1
%         - legend_font_size (int): font size for legend; default is axis_font_size
%         - legend_location (char): location of legend within plot; any Matlab designator accepted; default is 'Best'
%         - legend_interpreter (char): interpreter for record labels in legend, [] (default) uses system default, alternatively 'none','tex','latex'
%         - legend_tags (char): single string that must be present in a dataset's tag for inclusion in a legend, default is 'set';
%         - set_tags (char or cell array of char): the 'tags' field applied to the plot of each record, for selecting which records appear in legend, default is  'set 1', etc.
%         - if_box (int): 1 to include the bounding box in a 3d plot, 0 to omit; default is 1
%         - if_grid (int): 1 to include the grid, 0 to omit; default is 1
%         - if_finalize (int): 1 to apply view, legend, labels, and perpendiculars, 0 to omit; default is 1
%
%        - if_warn (int): 1 to display warnings related to plot configurations, 0 to suppress; default is 1
%
%     - opts_check (struct): options for consistency checking, with field
%
%         - if_warn (int): 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
% 
% Returns:
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_disp (struct): aux.opts_disp, with defaults and overrides filled in, including fig_handle (handle to the figure) and axis_handles (handles to the subplot axes)
%
% Note: Note regarding customization
%     - The default figure position can be changed by editing the line containing generic.opts_disp.fig_position in `rs_aux_defaults_define`, running it once, and saving the workspace as rs_aux_defaults.mat.
%     - The default font size for axis labels can be changed by editing the line containing generic.opts_disp.axis_font_size in `rs_aux_defaults_define`, running it  once, and saving the workspace as rs_aux_defaults.mat.
%     - The default prefix for the axis label can be changed by editing the line containing generic.opts_disp.axis_label_prefix in `rs_aux_defaults_define`, running it  once, and saving the workspace as rs_aux_defaults.mat.
%
% Note: Note regarding plot formatting
%     - By default, each record is plotted with a different color, a solid dot marker, and no connecting lines. 
%     - These choices can be changed by 'set_colors', 'set_markers',  'set_markersizes', 'set_filled', 'set_colors_filled', 'set_alphas' (but alpha blending may not be availble on all systems).
%     - The above specifiers can be singletons or cell arrays, and are indexed by the record position in data_in.  If there are more records than specifiers, the specifiers are cycled.
%     - The data points of each record are, by default, not interconnected. This can be changed with the 'connect_data*' options.
%     - The data points between records are, by default, not interconnected . This can be changed with the 'connect_sets*' options.
%
%  See also: RS_DISP_ENH_COORDSETS, RS_CHECK_COORDSETS, RS_GET_COORDSETS, RS_ALIGN_COORDSETS, RS_KNIT_COORDSETS, RS_PLOT_STYLE.
%
if (nargin<=1)
    aux=struct;
end
%fields that will be made into cells if singletons
make_cell={'set_colors','set_colors_filled','set_markers','connect_data_linestyles','set_labels','set_tags','legend_tags','axis_view','connect_sets_linestyles','connect_sets_colors',...
    'callout_colors','callout_linestyles','axis_labels','perp_data_linestyles'};
trunc_pad={'set_offsets','set_offsets_margin_amount','set_offsets_margin_fraction'}; %fields that are truncated or padded to have dim 2 length = dim_select
coords_together_allowed=[2 3]; %how many coords can be plotted together -eventually could include >=4
coords_together_default=[2 3]; %how many coords are plotted together by default
xyzlim={'XLim','YLim','ZLim'};
datapoint_tag='*datapoint*'; %string to identify data points
%
aux=filldefault(aux,'opts_check',struct);
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
%check consistency and get available stimuli, dimensions, typenames
%
aux_out=struct;
%
%prior to consistency checking, see if we need to allow for typenames to vary
%
data_label_typenames_vary=0;
if isfield(aux,'opts_disp')
    if isfield(aux.opts_disp,'data_label_typenames_vary')
        data_label_typenames_vary=aux.opts_disp.data_label_typenames_vary;
    end
end
data_in_check=data_in;
if data_label_typenames_vary
    for iset=1:length(data_in.sas)
        for istim=1:length(data_in.sas{iset}.typenames)
            data_in_check.sas{iset}.typenames{istim}=sprintf('stim_%1.0f',istim);
        end
    end
end
%
check=rs_check_coordsets(data_in_check,aux.opts_check);
%
aux_out.warnings=check.warnings;
aux_out.warn_bad=check.warn_bad;
nsets=check.nsets;
nstims_each=check.nstims_each;
dim_list_each=check.dim_list_each;
dim_list_union=check.dim_list_union;
dim_list_inter=check.dim_list_inter;
typenames_each=check.typenames_each;
typenames_union=check.typenames_union;
typenames_inter=check.typenames_inter;
%
nstims=min(nstims_each); %all nstims_each should be identical
%
%set up sub-structure options
aux=filldefault(aux,'opts_disp',struct); %options for this module (psg_template)
aux.opts_disp=filldefault(aux.opts_disp,'if_warn',1);
aux.opts_disp=filldefault(aux.opts_disp,'if_finalize',1);
%
aux.opts_disp=filldefault(aux.opts_disp,'fig_handle',[]);
aux.opts_disp=filldefault(aux.opts_disp,'axis_handles',[]);
aux.opts_disp=filldefault(aux.opts_disp,'axis_view',{3});
aux.opts_disp=filldefault(aux.opts_disp,'axis_equal',1);
aux.opts_disp=filldefault(aux.opts_disp,'axis_range','tight');
aux.opts_disp=filldefault(aux.opts_disp,'axis_range_list',[0 1]);
aux.opts_disp=filldefault(aux.opts_disp,'axis_labels',cell(0));
%
aux.opts_disp=filldefault(aux.opts_disp,'dim_select',max(intersect(coords_together_default,dim_list_inter)));
aux.opts_disp=filldefault(aux.opts_disp,'set_select',[1:nsets]);
aux.opts_disp=filldefault(aux.opts_disp,'set_offsets',zeros(1,aux.opts_disp.dim_select));
aux.opts_disp=filldefault(aux.opts_disp,'set_offsets_margin_amount',ones(1,aux.opts_disp.dim_select));
aux.opts_disp=filldefault(aux.opts_disp,'set_offsets_margin_fraction',zeros(1,aux.opts_disp.dim_select));
aux.opts_disp=filldefault(aux.opts_disp,'set_offsets_coordchoices','all');
%
aux.opts_disp=filldefault(aux.opts_disp,'coord_group_size',min(aux.opts_disp.dim_select,max(coords_together_default)));
aux.opts_disp=filldefault(aux.opts_disp,'coord_group_method','all');
aux.opts_disp=filldefault(aux.opts_disp,'fig_name',sprintf('dimension %2.0f',aux.opts_disp.dim_select));
%
aux.opts_disp=filldefault(aux.opts_disp,'set_colors',{'k','b','c','m','r','y','g'});
aux.opts_disp=filldefault(aux.opts_disp,'set_markers',{'.'});
aux.opts_disp=filldefault(aux.opts_disp,'set_markersizes',8);
aux.opts_disp=filldefault(aux.opts_disp,'set_filled',0);
aux.opts_disp=filldefault(aux.opts_disp,'set_colors_filled',aux.opts_disp.set_colors);
aux.opts_disp=filldefault(aux.opts_disp,'set_alphas',1);
%
aux.opts_disp=filldefault(aux.opts_disp,'data_show_method','all');
aux.opts_disp=filldefault(aux.opts_disp,'data_show_list',[]);
aux.opts_disp=filldefault(aux.opts_disp,'data_label_method','all');
aux.opts_disp=filldefault(aux.opts_disp,'data_label_list',[]);
aux.opts_disp=filldefault(aux.opts_disp,'data_label_setsel_method','first');
aux.opts_disp=filldefault(aux.opts_disp,'data_label_setsel_list',[]);
aux.opts_disp=filldefault(aux.opts_disp,'data_label_interpreter',[]);
aux.opts_disp=filldefault(aux.opts_disp,'data_label_typenames_vary',0);
%
aux.opts_disp=filldefault(aux.opts_disp,'callout_amount',0);
aux.opts_disp=filldefault(aux.opts_disp,'callout_colors',{'k'});
aux.opts_disp=filldefault(aux.opts_disp,'callout_linestyles',{'-.'});
aux.opts_disp=filldefault(aux.opts_disp,'callout_linewidths',1);
aux.opts_disp=filldefault(aux.opts_disp,'callout_center','centroid');
%
aux.opts_disp=filldefault(aux.opts_disp,'connect_data_method','none');
aux.opts_disp=filldefault(aux.opts_disp,'connect_data_list',[]);
aux.opts_disp=filldefault(aux.opts_disp,'connect_data_linestyles',{'none'});
aux.opts_disp=filldefault(aux.opts_disp,'connect_data_linewidths',1);
%
aux.opts_disp=filldefault(aux.opts_disp,'connect_sets_method','none');
aux.opts_disp=filldefault(aux.opts_disp,'connect_sets_list',[]);
aux.opts_disp=filldefault(aux.opts_disp,'connect_sets_data_method','all');
aux.opts_disp=filldefault(aux.opts_disp,'connect_sets_data_list',[]);
aux.opts_disp=filldefault(aux.opts_disp,'connect_sets_linestyles','-');
aux.opts_disp=filldefault(aux.opts_disp,'connect_sets_linewidths',1);
aux.opts_disp=filldefault(aux.opts_disp,'connect_sets_color_mode','split');
aux.opts_disp=filldefault(aux.opts_disp,'connect_sets_colors',[]);
%
aux.opts_disp=filldefault(aux.opts_disp,'perp_data_dims',0);
aux.opts_disp=filldefault(aux.opts_disp,'perp_data_linestyles','-');
aux.opts_disp=filldefault(aux.opts_disp,'perp_data_linewidths',1);
aux.opts_disp=filldefault(aux.opts_disp,'perp_data_usemarkers',1);
%
aux.opts_disp=filldefault(aux.opts_disp,'if_box',1);
aux.opts_disp=filldefault(aux.opts_disp,'if_grid',1);
aux.opts_disp=filldefault(aux.opts_disp,'if_legend',1);
aux.opts_disp=filldefault(aux.opts_disp,'legend_location','Best');
aux.opts_disp=filldefault(aux.opts_disp,'legend_interpreter',[]);
aux.opts_disp=filldefault(aux.opts_disp,'legend_tags',{'set'});
%
aux=rs_aux_customize(aux,'rs_disp_coordsets');
%quantities dependent on overall defaults
aux.opts_disp=filldefault(aux.opts_disp,'axis_label_font_size',aux.opts_disp.axis_font_size);
aux.opts_disp=filldefault(aux.opts_disp,'legend_font_size',aux.opts_disp.axis_font_size);
aux.opts_disp=filldefault(aux.opts_disp,'data_label_font_size',aux.opts_disp.axis_font_size);
%
%set up other defaults and check consistency
%
disp_msgs=[]; %for warnings encountered during plots
x=aux.opts_disp; %for convenience
switch x.coord_group_method %determine coordinate groups
    case 'all'
        x.coord_groups=nchoosek([1:x.dim_select],x.coord_group_size);
    case 'keeplow'
        x.coord_groups=[repmat([1:x.coord_group_size-1],x.dim_select-x.coord_group_size+1,1),[x.coord_group_size:x.dim_select]'];
    case 'keepone'
        coords_group_list_hi=nchoosek([2:x.dim_select],x.coord_group_size-1);
        x.coord_groups=[ones(size(coords_group_list_hi,1),1) coords_group_list_hi];
    case 'rolling'
        x.coord_groups=mod(repmat([0:x.coord_group_size-1],x.dim_select,1)+repmat([0:x.dim_select-1]',1,x.coord_group_size),x.dim_select)+1;
    case 'onlylowest'
        x.coord_groups=[1:x.coord_group_size];
    case 'list'
    otherwise
        wmsg=sprintf('grouping method (%s) not recognized; only lowest coords shown',x.coord_group_method);
        x.coord_groups=[1:x.coord_group_size];
        aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
end
if x.coord_group_size~=size(x.coord_groups,2)
    wmsg=sprintf('number of columnns in coord_groups (%3.0f) and coord_group_size (%3.0f) is inconsistent',size(x.coord_groups,2),x.coord_group_size);
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',x.if_warn));
end
if ~ismember(x.coord_group_size,coords_together_allowed)
    wmsg=sprintf('cannot plot groups of %3.0f coordinates on same axis',x.coord_group_size);
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',x.if_warn));
end
if any(x.coord_groups(:)<=0) | any(x.coord_groups(:)>x.dim_select)
    wmsg=sprintf('some specified dimensions are out of bounds for the dimension plotted (%2.0f)',x.dim_select);
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',x.if_warn));
end
if ~isempty(setdiff(x.set_select,[1:nsets]))
    wmsg=sprintf('some selections in set_select are out of bounds ([1:%1.0f], and ignored )',nsets);
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
    x.set_select=intersect(x.set_select,[1:nsets]);
end
ngroups=size(x.coord_groups,1);
ngroups_aug=ngroups;
if x.if_legend==-1
    ngroups_aug=ngroups+1;
end
%parse set_offsets
for ifn=1:length(trunc_pad) %trunc_pad={'set_offsets','set_offsets_margin_amount','set_offsets_margin_fraction'}; %fields that are truncated or padded to have dim 2 length = dim_select
    tp=trunc_pad{ifn};
    if isnumeric(x.(tp))
        needcols=[1 x.dim_select];
        needcols_msg=sprintf('1 to %1.0f',x.dim_select);
        %
        if_okcols=double((size(x.(tp),2)>=min(needcols)) & (size(x.(tp),2)<=max(needcols)));
        if if_okcols==0
            wmsg=sprintf('number of columns in %s (%1.0f) is inconsistent with number needed (%s); truncated or padded',tp,size(x.(tp),2),needcols_msg);
            aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
        end
        if size(x.(tp),2)>x.dim_select
            x.(tp)=x.(tp)(:,[1:x.dim_select]);
        end
        if size(x.(tp),2)<x.dim_select
            x.(tp)=cat(2,x.(tp),zeros(size(x.(tp),1),x.dim_select-size(x.(tp),2)));
        end
    end
end
if ~isnumeric(x.set_offsets)    
    %compute min and max of each coord in each set
    mins=zeros(nsets,x.dim_select);
    maxs=zeros(nsets,x.dim_select);
    for iset=1:nsets
        mins(iset,:)=min(data_in.ds{iset}{x.dim_select},[],1);
        maxs(iset,:)=max(data_in.ds{iset}{x.dim_select},[],1);
    end
    switch x.set_offsets
        case 'margin_amount' %space by an absolute amount
            upper=-Inf;
            x.set_offsets=zeros(nsets,x.dim_select);
            for iset=1:nsets
                if ismember(iset,x.set_select)
                    if (upper>-Inf) %have we encountered any sets
                        x.set_offsets(iset,:)=x.set_offsets_margin_amount+upper-mins(iset,:);
                        upper=x.set_offsets(iset,:)+maxs(iset,:);
                    else
                        upper=maxs(iset,:);
                    end
                end
            end
        case 'margin_fraction' %space by a fraction of the average size of the spans
            upper=-Inf;
            x.set_offsets=zeros(nsets,x.dim_select);
            for iset=1:nsets
                if ismember(iset,x.set_select)
                    if (upper>-Inf) %have we encountered any sets
                        meansize=(maxs(iset_prev,:)-mins(iset_prev,:)+maxs(iset,:)-mins(iset,:))/2; %mean span of current and previous set
                        x.set_offsets(iset,:)=x.set_offsets_margin_fraction.*meansize+upper-mins(iset,:);
                        upper=x.set_offsets(iset,:)+maxs(iset,:);
                        iset_prev=iset;
                    else
                        upper=maxs(iset,:);
                        iset_prev=iset;
                    end
                end
            end
      otherwise
            wmsg=sprintf('specification of offset (%s) not recognized; no offset used',x.set_offsets);
            aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
            x.set_offsets=zeros(1,x.dim_select);
    end
    x.set_offsets=x.set_offsets-repmat(mean(x.set_offsets(x.set_select,:),1),nsets,1); %center around zero
    %now process set_offsets_coordchoices:  which coordinates are offset in each coordinate group
    if ~ischar(x.set_offsets_coordchoices) %only force numeric arrays into cells
        if ~iscell(x.set_offsets_coordchoices)
            x.set_offsets_coordchoices={x.set_offsets_coordchoices};
        end
        off_choices=x.set_offsets_coordchoices;
    else %string values cannot be cells
        if iscell(x.set_offsets_coordchoices)
            x.set_offsets_coordchoices=x.set_offsets_coordchoices{1};
        end
        switch x.set_offsets_coordchoices
            case 'all'
                off_choices=num2cell(x.coord_groups,2);
            case 'first'
                off_choices=(num2cell(x.coord_groups(:,1)))';
            case 'last'
                off_choices=(num2cell(x.coord_groups(:,end)))';
            otherwise
                wmsg=sprintf('specification of offset coordinate choices (%s) not recognized; all coordinates offset',x.set_offsets_coordchoices);
                aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
                off_choices{1}=[1:x.dim_select];
        end
    end
    %convert off_choices to [0,1] selection
    x.offsets_select=zeros(ngroups,x.dim_select); %will have 1's to select coordinates with offsets
    for kptr=1:ngroups
        k=1+mod(kptr-1,length(off_choices));
        if (any(off_choices{k}<1) | any(off_choices{k}>x.dim_select) | any(off_choices{k}~=round(off_choices{k}))) 
            wmsg=sprintf(' specification of offset coordinate choices non-integer or out of range ([1 %1.0f], all coordinates offset',x.dim_select);
            aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
            off_choices{k}=[1:x.dim_select];
        end
        x.offsets_select(kptr,intersect(off_choices{k},x.coord_groups(kptr,:)))=1;
    end
else
    x.offsets_select=ones(ngroups,x.dim_select);
end
%
%set up axis scale
switch x.axis_range %check that it is 'tight','auto', or pairs of values
    case {'tight','auto'}
    case 'list'
        if (~isnumeric(x.axis_range_list) | size(x.axis_range_list,2)~=2)
            wmsg=sprintf('axis scale list must have two columns; tight scaling used');
            aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
            x.axis_range='tight';
            x.axis_range_list=[NaN NaN];
        end
    otherwise
        wmsg=sprintf('axis scale specifier (%s) not recognized, tight scaling used',x.axis_range);
        aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
        x.axis_range='tight';
        x.axis_range_list=[NaN NaN];
end
%set up data show and label params
[x.data_show_list,wmsg]=rs_disp_parse_listmethod(x.data_show_method,[1:nstims],x.data_show_list,'specification of data points to show');
if ~isempty(wmsg)
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
end
[x.data_label_list,wmsg]=rs_disp_parse_listmethod(x.data_label_method,[x.data_show_list],x.data_label_list,'specification of data points to label');
if ~isempty(wmsg)
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
end
[x.data_label_setsel_list,wmsg]=rs_disp_parse_listmethod(x.data_label_setsel_method,x.set_select,x.data_label_setsel_list,'specification of sets to label');
if ~isempty(wmsg)
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
end
%set up params for connecting points within sets
[x.connect_data_list,wmsg]=rs_disp_parse_pairmethod(x.connect_data_method,x.data_show_list,x.connect_data_list,'specification of data points to connect within a set');
if ~isempty(wmsg)
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
end
if ~isempty(x.connect_data_list) %convert a set of pairs to a set of chains to simplify plotting
    x.connect_data_chains=pairs2chains(x.connect_data_list);
else
    x.connect_data_chains=[];
end
%set up params for connecting points across sets
[x.connect_sets_list,wmsg]=rs_disp_parse_pairmethod(x.connect_sets_method,x.set_select,x.connect_sets_list,'specification of sets to connect');
if ~isempty(wmsg)
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
end
if strcmp(x.connect_sets_data_method,'labeled')
    x.connect_sets_data_method=x.data_label_method;
    x.connect_sets_data_list=x.data_label_list;
end
[x.connect_sets_data_list,wmsg]=rs_disp_parse_listmethod(x.connect_sets_data_method,[x.data_show_list],x.connect_sets_data_list,'specification of data points to connect between sets');
if ~isempty(wmsg)
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
end
%
%ensure that certain options are cell in a column, or cell array of triplets
for imc=1:length(make_cell)
    mc=make_cell{imc};
    if ~isempty(strfind(mc,'color'))
        x.(mc)=rs_disp_colorfix(x.(mc));
    end
end
%set up connection colors
if size(x.connect_sets_list,1)>0
    connect_sets_list_mod=mod(x.connect_sets_list-1,length(x.set_colors))+1;
    switch x.connect_sets_color_mode
        %connect_sets_list_mod=mod(x.connect_sets_list-1,length(x.set_colors))+1;
        case 'first'
            x.connect_sets_colors=x.set_colors(connect_sets_list_mod(:,1));
        case 'last'
            x.connect_sets_colors=x.set_colors(connect_sets_list_mod(:,2));
        case 'split'
            x.connect_sets_colors=[x.set_colors(connect_sets_list_mod(:,1)),x.set_colors(connect_sets_list_mod(:,2))];
        case 'list'
        otherwise
            wmsg=sprintf('connect color mode (%s) not recognized, connections ignored',x.connect_sets_method);
            x.connect_sets_list=[];
            aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
    end
else
    x.connect_sets_list=[];
end
%set up labels
if ~isfield(x,'set_labels')
    x.set_labels=cell(nsets,1);
    for k=1:nsets
        x.set_labels{k}=sprintf('set %1.0f',k);
    end
end
if ~isfield(x,'set_tags')
    x.set_tags=cell(nsets,1);
    for k=1:nsets
        x.set_tags{k}=sprintf('set %1.0f',k);
    end
end
%ensure that certain options are cells
for imc=1:length(make_cell)
    mc=make_cell{imc};
    if ~iscell(x.(mc))
        x.(mc)={x.(mc)};
    end
end
%set_labels and set_tags cannot be empty
for k=1:length(x.set_labels)
    if isempty(x.set_labels{k})
        x.set_labels{k}=sprintf('set %1.0f',k);
    end
end
for k=1:length(x.set_tags)
    if isempty(x.set_tags{k})
        x.set_tags{k}=sprintf('set %1.0f',k);
    end
end
%use set colors for callout colors if requested
if strmatch(x.callout_colors{1},'set_colors')
    x.callout_colors=x.set_colors;
end
%
naxis_handles=length(x.axis_handles);
if naxis_handles>0 & naxis_handles~=ngroups_aug
    wmsg=sprintf('number of axes (subplots) supplied (%3.0f) does not match number of axes needed (%3.0f)',naxis_handles,ngroups_aug);
    aux_out=rs_warning(wmsg,double(naxis_handles<ngroups_aug),setfield(aux_out,'if_warn',x.if_warn));
end
%
if aux_out.warn_bad==0
    %set up styles for each component:  dataset, connections between data in a set, callouts, connections between sets, perpendiculars from data
    set_styles=struct;
    set_styles.colors=x.set_colors;
    set_styles.markers=x.set_markers;
    set_styles.markersizes=x.set_markersizes;
    set_styles.linestyles={'none'};
    set_styles.linewidths=1;
    set_styles.colors_filled=x.set_colors_filled;
    set_styles.filled=x.set_filled;
    set_styles.alphas=x.set_alphas;
    %
    connect_data_styles=struct;
    connect_data_styles.colors=x.set_colors;
    connect_data_styles.markers={'none'};
    connect_data_styles.markersizes=8;
    connect_data_styles.linestyles=x.connect_data_linestyles;
    connect_data_styles.linewidths=x.connect_data_linewidths;
    %
    callout_styles=struct;
    callout_styles.colors=x.callout_colors;
    callout_styles.markers={'none'};
    callout_styles.markersizes=8;
    callout_styles.linestyles=x.callout_linestyles;
    callout_styles.linewidths=x.callout_linewidths;
    %
    connect_set_styles=struct;
    connect_set_styles.colors={'k'};
    connect_set_styles.markers={'none'};
    connect_set_styles.markersizes=8;
    connect_set_styles.linestyles=x.connect_sets_linestyles;
    connect_set_styles.linewidths=x.connect_sets_linewidths;
    %
    if isempty(x.fig_handle)
        x.fig_handle=figure;
    else
        if x.fig_handle~=gcf
            figure(x.fig_handle);
        end
    end
    set(gcf,'Position',x.fig_position);
    set(gcf,'NumberTitle','off');
    set(gcf,'Name',x.fig_name);
    if naxis_handles==0
        fig_posit=get(gcf,'Position');
        [nrows,ncols]=nicesubp(ngroups_aug,fig_posit(4)/fig_posit(3)); %find an arrangement of rows and columns that fits the aspect ratio
        for igp=1:ngroups_aug
            x.axis_handles{igp}=subplot(nrows,ncols,igp);
        end
    end
    callout_center_ok=0;
    if isnumeric(x.callout_center)
        if length(x.callout_center)==x.dim_select
            callout_center_ok=1;
        end
    else
        switch x.callout_center
            case 'centroid'
                callout_center_ok=1;
            case 'zero'
                callout_center_ok=1;
        end
    end
    if callout_center_ok==0
        wmsg='callout_center specification not recognized, zero used';
        aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',x.if_warn));
        x.callout_center='zero';
    end
    %
    for igp_aug=1:ngroups_aug
        haxis=x.axis_handles{igp_aug};
        igp=mod(igp_aug-1,ngroups)+1; %if igp_aug=ngroup+1 (if_legend=-1) then igp=1 but it is plotted in a new subplot
        subplot(haxis);
        set(gca,'FontSize',x.axis_font_size);
        cg=x.coord_groups(igp,:);
        %plot points
        for isetptr=1:length(x.set_select)
            %
            [z,z_all,k]=rs_disp_coordsets_tuples(data_in,isetptr,x,igp); %get the tuples of all the points and the plotted ponts
            %
            [hline,plotstyle_used,opts_plotstyle_used]=rs_disp_doplot(z,k,set_styles);
            disp_msgs=strvcat(disp_msgs,opts_plotstyle_used.msgs);
            set(hline,'Tag',cat(2,x.set_tags{1+mod(k-1,length(x.set_tags))},' ',datapoint_tag)); %so that we can use it to drop perpendiculars when finalized
            set(hline,'DisplayName',x.set_labels{1+mod(k-1,length(x.set_labels))});
            %connect requested points
            if ~isempty(x.connect_data_chains)
                for ichain=1:length(x.connect_data_chains)
                    [hline,plotstyle_used,opts_plotstyle_used]=rs_disp_doplot(z_all(x.connect_data_chains{ichain},:),k,connect_data_styles);
                    disp_msgs=strvcat(disp_msgs,opts_plotstyle_used.msgs);
                    set(hline,'Tag',sprintf('ds %2.0f chain %2.0f',k,ichain));
                    set(hline,'DisplayName',x.set_labels{1+mod(k-1,length(x.set_labels))});
                end
            end
            %label?
            if ismember(k,x.data_label_setsel_list)
                zcallout=z; 
                if x.callout_amount>0 %compute position of call-out for every data point shown
                    callout_center=zeros(1,x.coord_group_size);
                    if isnumeric(x.callout_center)
                        callout_center=x.callout_center(1,cg);
                    else
                        switch x.callout_center
                            case 'centroid'
                                callout_center=mean(z_all,1,'omitnan');
                        end
                    end
                    dists=sqrt(sum((z_all-repmat(callout_center,size(z_all,1),1)).^2,2,'omitnan')); %distances from callout_center
                    rmsdist=sqrt(mean(dists.^2)); %rms distances from callout_center
                    dists(dists==0)=1;
                    for lab=1:size(x.data_show_list,1)
                        zcallout(lab,:)=callout_center+(zcallout(lab,:)-callout_center)*(1+x.callout_amount*rmsdist/dists(x.data_show_list(lab)));
                    end
                end
                typenames=data_in.sas{k}.typenames;
                for ipt_ptr=1:length(x.data_label_list)
                    ipt=x.data_label_list(ipt_ptr);
                    ipt_rel=find(x.data_show_list==ipt); %which of the selected points is being labelled
                    switch x.coord_group_size
                        case 2
                            ht=text(zcallout(ipt_rel,1),zcallout(ipt_rel,2),typenames{ipt});
                        case 3
                            ht=text(zcallout(ipt_rel,1),zcallout(ipt_rel,2),zcallout(ipt_rel,3),typenames{ipt});
                    end %coord group size
                    set(ht,'FontSize',x.data_label_font_size);
                    if ~isempty(x.data_label_interpreter)
                        set(ht,'Interpreter',x.data_label_interpreter);
                    end
                    if x.callout_amount>0
                        [hcallout,plotstyle_used,opts_plotstyle_used]=rs_disp_doplot(cat(1,z(ipt_rel,:),zcallout(ipt_rel,:)),k,callout_styles);
                        disp_msgs=strvcat(disp_msgs,opts_plotstyle_used.msgs);
                    end
                end %ipt
            end %set to label?
        end %isetptr
        %set up view and axis labels
        n_axis_labels=length(x.axis_labels);
        ax_labels=cell(1,x.coord_group_size);
        for ix=1:x.coord_group_size
            if isempty(x.axis_labels) | n_axis_labels==0;
                ax_labels{ix}=sprintf('%s %1.0f',x.axis_label_prefix,cg(ix));
            else
                ax_labels{ix}=x.axis_labels{1+mod(ix-1,n_axis_labels)};
            end
        end
        if x.if_finalize
            if x.coord_group_size<=3
                hl=xlabel(ax_labels{1});
                set(hl,'FontSize',x.axis_label_font_size);
                hl=ylabel(ax_labels{2});
                set(hl,'FontSize',x.axis_label_font_size);
            end
            if (x.coord_group_size==3)
                hl=zlabel(ax_labels{3});
                set(hl,'FontSize',x.axis_label_font_size);
                axis vis3d;
                index_view=1+mod(igp-1,length(x.axis_view));
                view(x.axis_view{index_view});
            end
        end
        %connections between sets
        for ic=1:size(x.connect_sets_list,1)
            cset=x.connect_sets_list(ic,:);
            if all(ismember(cset,x.set_select))
                data_connect_list=intersect(x.data_show_list,x.connect_sets_data_list); %find the points that are shown and also those selected for inter-set connection
                endpoints=zeros(length(data_connect_list),x.coord_group_size,2);
                for iz=1:2
                    endpoints(:,:,iz)=data_in.ds{cset(iz)}{x.dim_select}(data_connect_list,cg);
                    ko=mod(cset(iz)-1,size(x.set_offsets,1))+1;
                    endpoints(:,:,iz)=endpoints(:,:,iz)+repmat(x.set_offsets(ko,cg).*x.offsets_select(igp,cg),size(endpoints,1),1,1); %add the offset
                end
                midpoints=mean(endpoints,3);
                if ~strcmp(x.connect_sets_color_mode,'split')
                    connect_set_styles.colors=x.connect_sets_colors;
                    for istim=1:length(data_connect_list)
                        [hconnect,plotstyle_used,opts_plotstyle_used]=rs_disp_doplot([endpoints(istim,:,1);endpoints(istim,:,2)],ic,connect_set_styles);
                        disp_msgs=strvcat(disp_msgs,opts_plotstyle_used.msgs);
                    end
                else
                    for istim=1:length(data_connect_list)
                        for iseg=1:2
                            [hconnect,plotstyle_used,opts_plotstyle_used]=rs_disp_doplot([endpoints(istim,:,iseg);midpoints(istim,:)],ic,setfield(connect_set_styles,'colors',x.connect_sets_colors(:,iseg)));
                            disp_msgs=strvcat(disp_msgs,opts_plotstyle_used.msgs);
                        end %each segment
                    end %each stimulus
                end %split or not
            end %both sets are plotted
        end %each connection pair
        %box, view, axis scaling, legend
        if  x.if_finalize
            if (x.coord_group_size==3)
                if x.if_box
                    box on;
                else
                    box off;
                end
            end
            if x.if_grid
                grid on;
            else
                grid off;
            end
            if x.axis_equal
                axis equal;
            else
                axis normal;
            end
            switch x.axis_range
                case 'tight'
                    axis tight;
                case 'auto'
                    axis auto;
                case 'list'
                    for ic=1:x.coord_group_size
                        set(gca,xyzlim{ic},x.axis_range_list(1+mod(x.coord_groups(igp,ic)-1,size(x.axis_range_list,1)),:));
                    end
            end
            %perpendiculars: need to be plotted after axis range is established
            if (x.coord_group_size==3)
                for ic=1:x.coord_group_size
                    if_perp=x.perp_data_dims(1+mod(cg(ic)-1,length(x.perp_data_dims)));
                    axis_lims=get(gca,xyzlim{ic});
                    if if_perp~=0
                        hc=get(gca,'Children'); %look for data points via tags
                        z_foot=axis_lims((3-if_perp)/2); %foot of perp is min if if_perp=1, max if if_perp=-1
                        for ichild=1:length(hc)
                            if contains(hc(ichild).Tag,datapoint_tag)
                                xyz_data=[hc(ichild).XData;hc(ichild).YData;hc(ichild).ZData]';
                                %
                                perp_data_style=struct;
                                perp_data_style.linewidths=x.perp_data_linewidths;
                                %color taken from data
                                perp_data_style.colors={hc(ichild).Color};
                                %marker optionally used
                                perp_data_style.markersizes=hc(ichild).MarkerSize;
                                if x.perp_data_usemarkers
                                    perp_data_style.markers={hc(ichild).Marker};
                                else
                                    perp_data_style.markers={'none'};
                                end
                                if strcmp(x.perp_data_linestyles{1},'use_data')
                                    perp_data_style.linestyles={hc(ichild).LineStyle};
                                else
                                    perp_data_style.linestyles=x.perp_data_linestyles;
                                end
                                for iperp=1:size(xyz_data,1)
                                    z_to_perp=repmat(xyz_data(iperp,:),2,1);
                                    z_to_perp(2,ic)=z_foot;
                                    [hperp,plotstyle_used,opts_plotstyle_used]=rs_disp_doplot(z_to_perp,k,perp_data_style);
                                end %each point
                            end %each line with data
                        end
                    end %if_perp
                end %coordinate
            end
            %legend
            if (x.if_legend==1)
                need_legend=1;
            elseif (x.if_legend==-1)
                need_legend=(igp_aug==ngroups_aug);
            else
                need_legend=0;
            end
            if need_legend
                hc=get(haxis,'Children');
                tags_all=cell(length(hc),1);
                for ich=1:length(hc)
                    tags_all{ich}=get(hc(ich),'Tag');
                end
                hc_show=[];
                for itag=1:length(x.legend_tags)
                    hc_show=union(hc_show,strmatch(x.legend_tags{itag},tags_all)); %select the objects whose tags start with x.legend_tags
                end
                if ~isempty(hc_show)
                    h_legend=legend(hc(flipud(hc_show)),'Location',x.legend_location,'FontSize',x.legend_font_size);  %flipud since children appear to be added in reverse order
                    if ~isempty(x.legend_interpreter)
                        set(h_legend,'Interpreter',x.legend_interpreter);
                    end
                end
            end
        end %if_finalize
    end %igp_aug
end
%
aux_out=rs_warning(unique(disp_msgs,'rows'),0,setfield(aux_out,'if_warn',x.if_warn));
aux_out.opts_disp=x;
return
end

`rs_disp_enh_coordsets(data_in, aux, rays)`

aux_out=rs_disp_enh_coordsets(data_in,aux,rays) displayes one or more views of the coordinates in a dataset structure, with graphical enhancements: connecting stimuli along rays, in rings, and nearest neighbors. These enhancements depend on the availability of a ray structure, which specifyues rays (stimuli that lie on an approximate straight line from the origin) and rings (stimuli that lie in a plane at approximately equal distances from the origin), and nearest-neigbhbor pairs.

Input arguments:

data_in (struct) –
dataset structure to be processed, with fields
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record

aux (struct) –

auxiliary inputs, may be empty, with fields

opts_disp_enh (struct) – options for enhanced display, with fields
- if_points (int) – 1 to display each data point, 0 to suppress; default is 1; see notes below regarding points and rays
- if_rays (int) – 1 to connect data points along rays, 0 to suppress; default is 1 if 'rays' is non-empty, otherwise 0; see notes below regarding points and rays
- if_rings (int) – 1 to connect data points in rings, 0 to suppress; default is 1 if 'rays' is non-empty, otherwise 0
- if_nbrs (int) – 1 to connect nearest-neighbors, 0 to suppress; default is 1 if 'rays' is non-empty, otherwise 0
- if_nbrs_notsameray (int) – 1 to suppress nearest-neighbor connections if both points are on the same ray, or next to origin; 0 to connect all nearest neighbors; default is 1
- if_usetypenames (int) – how colors and symbols are determined; 1 (use 'typenames') is default; 0-> determine from stimulus coordinates, a field of data_in.sas{1} (search order: 'type_coords','btc_specoords','btc_augcoords', see rs_findrays for further details)
opts_tn2c (struct) – controls how stimulus labels (data_in.sas{:}.typenames) are mapped to colors and symbols; can be empty, see rs_typenames2colors for details

opts_disp (struct) – controls basic display; see rs_disp_coordsets for details. Most fields are of opts_disp are passed directly to rs_disp_coordsets, with unspecified fields in opts_disp filled with the defaults of rs_disp_coordsets. The following field values are inserted into opts_disp based on the fields of opts_disp_enh and the graphical element being drawn:

element:	points	rays	rings	neighbors
data_show_method	'all' [1]	'list'	'list'	'list'
data_label_method	'none'[1]	'last'	'none'[1]	'none'[1]
connect_data_method		'list'	'list'[1]	'list'
connect_data_linestyles	'none'[1]	'--' or '-'[2]	':' [1]	'-' [1]
set_markers		[3]	'none'[1]	'none'[1]
set_tags		'rays'	'rings'	'nbrs'
callout_amount		0.5[1]
set_colors		per ray[3]
callout_colors		per ray[1]

[1]: if a value is supplied value in aux.opts_disp, it is not overridden
[2]: line style depends on whether the ray is negative (first option) or positive (second option)
[3]: set markers and colors determined by rs_typenames2colors

rays (struct) –

a ray structure, ordinarily created by rs_findrays. If empty or omitted the enhanced graphical elements will not be displayed.

Output arguments:

aux_out ( struct ) –
auxiliary outputs and parameter values used
- opts_disp (struct) – aux.opts_disp, with defaults and overrides filled in
- opts_disp_enh (struct) – aux.opts_disp_enh, with defaults filled in
- opts_t2nc (struct) – aux.opts_t2nc, with defaults filled in
- points (struct) – aux_out from rs_disp_coordsets for display of points; omitted if opts_disp_enh.if_points=0
- rays (cell array of struct) – aux_out from rs_disp_coordsets for display of each ray; rays{:,isign} corresponds to display of each ray in the negative direction (isign=1) and the positive direction, (isign=2); omitted if opts_disp_enh.if_rays=0;
- rings (cell array of struct) – aux_out from rs_disp_coordsets for display of each ring; omitted if no rings are displayed or opts_disp_enh.if_rings=0
- nbrs (struct) – aux_out from rs_disp_coordsets for connections between neighbors; omitted if no connections are displayed or opts_disp_enh.if_nbrs=0

Note regarding points and rays

Rays are plotted before data points, so that data points overlay the rays and can be color-coded by set.
Legend behavior: If opts_disp_enh.if_rays=1 and opts_disp_enh.if_points=0, legend is the ray label; otherwise legend is set label.

Source code in src/rs_disp_enh_coordsets.m

function aux_out=rs_disp_enh_coordsets(data_in,aux,rays)
% aux_out=rs_disp_enh_coordsets(data_in,aux,rays) displayes one or more
% views of the coordinates in a `dataset structure`, with graphical enhancements: connecting stimuli along rays, in rings, and nearest neighbors.
% These enhancements depend on the availability of a `ray structure`, which specifyues rays (stimuli that lie on an approximate straight line from the origin) and
% rings (stimuli that lie in a plane at approximately equal distances from the origin), and nearest-neigbhbor pairs.
% 
%
% Args:
%   data_in (struct): `dataset structure` to be processed, with fields
%
%     - ds (cell array): `coordinate structure`, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
%     - sas (cell array): `stimulus metadata structure`, sas{k} is the stimulus metadata for the kth record
%     - sets (cell array): `set metadata structure`, sets{k} is the response metadata for the kth record
%
%   aux (struct): auxiliary inputs, may be empty, with fields
%
%     - opts_disp_enh (struct): options for enhanced display, with fields
%
%         - if_points (int): 1 to display each data point, 0 to suppress; default is 1; see notes below regarding points and rays
%         - if_rays (int): 1 to connect data points along rays, 0 to suppress; default is 1 if 'rays' is non-empty, otherwise 0; see notes below regarding points and rays
%         - if_rings (int): 1 to connect data points in rings, 0 to suppress; default is 1 if 'rays' is non-empty, otherwise 0
%         - if_nbrs (int): 1 to connect nearest-neighbors, 0 to suppress; default is 1 if 'rays' is non-empty, otherwise 0
%         - if_nbrs_notsameray (int): 1 to suppress nearest-neighbor connections  if both points are on the same ray, or next to origin; 0 to connect all nearest neighbors; default is 1
%         - if_usetypenames (int): how colors and symbols are determined; 1 (use 'typenames') is default; 0-> determine from stimulus coordinates, a field of data_in.sas{1} (search order: 'type_coords','btc_specoords','btc_augcoords', see `rs_findrays` for further details)
%
%      - opts_tn2c (struct): controls how stimulus labels (data_in.sas{:}.typenames) are mapped to colors and symbols; can be empty, see `rs_typenames2colors` for details
% 
%      - opts_disp (struct): controls basic display; see `rs_disp_coordsets` for details. Most fields are of opts_disp are passed directly to `rs_disp_coordsets`, with unspecified fields in opts_disp filled with the defaults of `rs_disp_coordsets`. The following field values are inserted into opts_disp based on the fields of opts_disp_enh and the graphical element being drawn:
%
%
%          |                      element: | points       | rays              | rings        | neighbors    |
%          |-------------------------------|--------------|-------------------|--------------|--------------|
%          | data_show_method              | 'all' [1]    | 'list'            | 'list'       | 'list'       |
%          | data_label_method             | 'none'[1]    | 'last'            | 'none'[1]    | 'none'[1]    |
%          | connect_data_method           |              | 'list'            | 'list'[1]    | 'list'       |
%          | connect_data_linestyles       | 'none'[1]    | '--' or '-'[2]    | ':'  [1]     | '-'   [1]    |
%          | set_markers                   |              | [3]               | 'none'[1]    | 'none'[1]    |
%          | set_tags                      |              | 'rays'            | 'rings'      | 'nbrs'       |
%          | callout_amount                |              | 0.5[1]            |              |              |
%          | set_colors                    |              | per ray[3]        |              |              |
%          | callout_colors                |              | per ray[1]        |              |              |
%
%         - [1]: if a value is supplied value in aux.opts_disp, it is not overridden
%         - [2]: line style depends on whether the ray is negative (first option) or positive (second option)
%         - [3]: set markers and colors determined by `rs_typenames2colors` 
%
%
%   rays (struct): a `ray structure`, ordinarily created by `rs_findrays`. If empty or omitted the enhanced graphical elements will not be displayed.
%
% Returns:
%   aux_out (struct): auxiliary outputs and parameter values used
%
%     - opts_disp (struct): aux.opts_disp, with defaults and overrides filled in
%     - opts_disp_enh (struct): aux.opts_disp_enh, with defaults filled in
%     - opts_t2nc (struct): aux.opts_t2nc, with defaults filled in
%     - points (struct): aux_out from `rs_disp_coordsets` for display of points; omitted if opts_disp_enh.if_points=0
%     - rays (cell array of struct): aux_out from `rs_disp_coordsets` for display of each ray; rays{:,isign} corresponds to display of each ray in the negative direction (isign=1) and the positive direction, (isign=2); omitted if opts_disp_enh.if_rays=0;
%     - rings (cell array of struct): aux_out from `rs_disp_coordsets` for display of each ring; omitted if no rings are displayed or opts_disp_enh.if_rings=0
%     - nbrs (struct): aux_out from `rs_disp_coordsets` for connections between neighbors; omitted if no connections are displayed or opts_disp_enh.if_nbrs=0
%
% Note: Note regarding points and rays
%     - Rays are plotted before data points, so that data points overlay the rays and can be color-coded by set.
%     - Legend behavior: If opts_disp_enh.if_rays=1 and opts_disp_enh.if_points=0, legend is the ray label; otherwise legend is set label.
%
%  See also: RS_DISP_COORDSETS, RS_TYPENAMES2COLORS, RS_FINDRAYS, RS_PLOT_STYLE.
%
if (nargin<=1)
    aux=struct;
end
if (nargin<=2)
    rays=struct;
end
%
aux=filldefault(aux,'opts_disp_enh',struct());
aux=filldefault(aux,'opts_tn2c',struct());
aux=filldefault(aux,'opts_disp',struct());
%
aux.opts_disp_enh=filldefault(aux.opts_disp_enh,'if_points',1);
aux.opts_disp_enh=filldefault(aux.opts_disp_enh,'if_rays',1);
aux.opts_disp_enh=filldefault(aux.opts_disp_enh,'if_rings',0);
aux.opts_disp_enh=filldefault(aux.opts_disp_enh,'if_nbrs',1);
aux.opts_disp_enh=filldefault(aux.opts_disp_enh,'if_nbrs_notsameray',1);
aux.opts_disp_enh=filldefault(aux.opts_disp_enh,'if_usetypenames',1);
%
if aux.opts_disp_enh.if_usetypenames==0 %get coordinates
    opts_rays=getfield(rs_aux_customize(setfield(struct(),'opts_rays',struct()),'rs_findrays'),'opts_rays'); %get defaults for ray_reorder_ring and ray_plane_jit
    opts_rays=filldefault(opts_rays,'coord_names',getfield(psg_defopts,'coord_fields')); % was {'type_coords','btc_specoords','btc_augcoords'});
    [stim_coords,coord_names]=psg_type_coords_util(data_in.sas{1},opts_rays.coord_names);
end
%
aux.opts_tn2c.paradigm_type=data_in.sets{1}.paradigm_type; %so that rs_typenames2colors knows the paradigm
aux.opts_tn2c.if_usetypenames=aux.opts_disp_enh.if_usetypenames;
%
if isempty(rays) | isempty(fieldnames(rays))
    aux.opts_disp_enh.if_rays=0;
    aux.opts_disp_enh.if_rings=0;
    aux.opts_disp_enh.if_nbrs=0;
end
%ensure that a figure handle is open and that we plot into currenf figure
if ~isfield(aux.opts_disp,'fig_handle')
    aux.opts_disp.fig_handle=[];
end
if isempty(aux.opts_disp.fig_handle)
    aux.opts_disp.fig_handle=figure;
end
if aux.opts_disp.fig_handle~=gcf
    aux.opts_disp.fig_handle=gcf;
end
%
if aux.opts_disp_enh.if_rays %plot points along each ray, in designated colors
    %customize standard plot options for rays
    opts_disp_rays=aux.opts_disp;
    opts_disp_rays.data_show_method='list';
    opts_disp_rays.connect_data_method='chain';
    opts_disp_rays=filldefault(opts_disp_rays,'data_label_method','last');
    opts_disp_rays.set_tags='rays'; %to identify for legend
    if aux.opts_disp_enh.if_points==0
        opts_disp_rays.legend_tags='rays'; %to include in legend
    end
    opts_disp_rays=filldefault(opts_disp_rays,'callout_amount',0.5);
    if_callout_colors_supplied=double(isfield(opts_disp_rays,'callout_colors'));
    aux_used_rays=[];
    for iray=1:rays.nrays
        orig_ptr=min(find(rays.whichray==0));
        for isign=-1:2:1
            bidir=find(rays.whichray==iray);
            mults=rays.mult(bidir);
            sign_sel=find(sign(mults)==isign);
            if ~isempty(sign_sel)
                bidir_sel=bidir(sign_sel);
                mults_sel=mults(sign_sel);
                mb_sorted=sort([abs(mults_sel(:)),bidir_sel],1,'ascend');
                bidir_sorted=mb_sorted(:,2); %sorted in ascending order of magnitude
                opts_disp_rays.data_show_list=[orig_ptr bidir_sorted']; %add origin to the beginning
                %
                %[rgb,symb,vecs,opts_used]=psg_typenames2colors(data_in.sas{1}.typenames(bidir_sorted),aux.opts_tn2c); %get standard colors and symbols
                if aux.opts_disp_enh.if_usetypenames
                    [rgb,symb]=rs_typenames2colors(data_in.sas{1}.typenames(bidir_sorted),setfield(struct(),'opts_tn2c',aux.opts_tn2c));
                else
                    [rgb,symb]=rs_typenames2colors(stim_coords(bidir_sorted,:),setfield(struct(),'opts_tn2c',aux.opts_tn2c));
                end
                opts_disp_rays.set_colors=rgb;
                if ~if_callout_colors_supplied
                    opts_disp_rays.callout_colors=rgb;
                end               
                opts_disp_rays.set_markers=symb;
                switch isign
                    case 1
                        opts_disp_rays.connect_data_linestyles='-';
                    case -1
                        opts_disp_rays.connect_data_linestyles='--';
                end
                %
                if isign==-1 | iray<rays.nrays
                    opts_disp_rays.if_finalize=0;
                else
                    opts_disp_rays.if_finalize=1;
                end
                opts_disp_rays.set_labels=data_in.sas{1}.typenames(bidir_sorted(end)); %use typenames of last stimulus in the ray
                aux_out_ray=rs_disp_coordsets(data_in,setfield(struct,'opts_disp',opts_disp_rays));
                aux_out.rays{iray,(3+isign)/2}=aux_out_ray;
                aux.opts_disp=rs_disp_enh_hupdate(aux.opts_disp,aux_out_ray.opts_disp);
                if isempty(aux_used_rays) %to use for legends
                    aux_used_rays=aux_out_ray;
                end
            end %not empty
        end %sign
    end %iray
end
%
if aux.opts_disp_enh.if_points %plot individual points?
    %customize standard plot options for points
    opts_disp_points=aux.opts_disp;
    opts_disp_points=filldefault(opts_disp_points,'data_show_method','all');
    opts_disp_points=filldefault(opts_disp_points,'data_label_method','none');
    opts_disp_points=filldefault(opts_disp_points,'connect_data_linestyles','-');
    %
    aux_out.points=rs_disp_coordsets(data_in,setfield(struct(),'opts_disp',opts_disp_points));
    %update figure and axis handles so future plots are in same place
    aux.opts_disp=rs_disp_enh_hupdate(aux.opts_disp,aux_out.points.opts_disp);
end
%
if aux.opts_disp_enh.if_rings %connect rings
    %customize standard plot options for rings
    opts_disp_rings=aux.opts_disp;
    opts_disp_rings.data_show_method='list';
    opts_disp_rings.connect_data_method='list';
    opts_disp_rings=filldefault(opts_disp_rings,'set_markers','none');
    opts_disp_rings=filldefault(opts_disp_rings,'data_label_method','none');
    opts_disp_rings=filldefault(opts_disp_rings,'connect_data_linestyles',':');
    opts_disp_rings.set_tags='rings'; %to identify for legend
    %find which ring is the last non-empty ring so if_finalize can be used to speed up
    ring_nz=zeros(1,rays.nrings);
    for iring=1:rays.nrings
        ring_nz(iring)=length(rays.rings{iring}.coord_ptrs);
    end
    ring_last=max(find(ring_nz>0));
    for iring=1:rays.nrings
        ring_list=rays.rings{iring}.coord_ptrs;
        if ~isempty(ring_list)
            ring_list_offset=[ring_list(2:end) ring_list(1)];
            opts_disp_rings.data_show_list=ring_list;
            opts_disp_rings.connect_data_list=[ring_list(:),ring_list_offset(:)];
            if iring<ring_last
                opts_disp_rings.if_finalize=0;
            else
                opts_disp_rings.if_finalize=1;
            end
            %
            aux_out_ring=rs_disp_coordsets(data_in,setfield(struct,'opts_disp',opts_disp_rings));
            aux_out.rings{iring}=aux_out_ring;
            aux.opts_disp=rs_disp_enh_hupdate(aux.opts_disp,aux_out_ring.opts_disp);
         end %empty?
    end %iray
end
%
if aux.opts_disp_enh.if_nbrs %connect neighbors
   if rays.npairs>0
        %customize standard plot options for rings
        opts_disp_nbrs=aux.opts_disp;
        opts_disp_nbrs.data_show_method='list';
        opts_disp_nbrs.connect_data_method='list';
        opts_disp_nbrs=filldefault(opts_disp_nbrs,'set_markers','none');
        opts_disp_nbrs=filldefault(opts_disp_nbrs,'data_label_method','none');
        opts_disp_nbrs=filldefault(opts_disp_nbrs,'connect_data_linestyles','-');
        opts_disp_nbrs.set_tags='nbrs'; %so that this will not be in legend
        pairs=rays.pairs;
        if aux.opts_disp_enh.if_nbrs_notsameray
            pair_rayid=reshape(rays.whichray(pairs),size(pairs));
            pair_keep=double(pair_rayid(:,1)~=pair_rayid(:,2)); %note, this will keep if both are NaN
            pair_keep=and(pair_keep,all(pair_rayid~=0,2)); %exclude rays that include the origin
            pairs=pairs(find(pair_keep),:);
        end
        if ~isempty(pairs)
            opts_disp_nbrs.data_show_list=unique(pairs(:));
            opts_disp_nbrs.connect_data_list=pairs;
            opts_disp_nbrs.set_markers={'none'};
            aux_out_nbrs=rs_disp_coordsets(data_in,setfield(struct,'opts_disp',opts_disp_nbrs));
            aux_out.nbrs=aux_out_nbrs;
            aux.opts_disp=rs_disp_enh_hupdate(aux.opts_disp,aux_out_nbrs.opts_disp);
        end
    end %empty?
end
%
%if rays but not points are plotted, then clean up legend: 
% ray label will have been repeated for each dataset
%
if aux.opts_disp_enh.if_rays==1 & aux.opts_disp_enh.if_points==0 & ~isempty(aux_used_rays)
    x=aux_used_rays.opts_disp; %these are the options used for plotting rays
    for iax=1:length(aux.opts_disp.axis_handles)
        haxis=aux.opts_disp.axis_handles{iax};
        legh=get(haxis,'Legend');
        if ~isempty(legh)
            subplot(haxis);
            hc=get(haxis,'Children');
            tags_all=cell(length(hc),1);
            display_all=cell(length(hc),1);
            for ich=1:length(hc)
                tags_all{ich}=get(hc(ich),'Tag');
                display_all{ich}=get(hc(ich),'DisplayName');
            end
            hc_show=strmatch('rays',tags_all); %select the objects whose tags start with x.legend_tags
            if ~isempty(hc_show)
                %find the unique display names, to eliminate duplicates across datasets
                display_unique=unique(display_all(hc_show));
                hc_show_unique=zeros(length(display_unique),1);
                for iu=1:length(display_unique)
                    hc_show_unique(iu)=hc_show(min(strmatch(display_unique{iu},display_all(hc_show),'exact'))); 
                end
                h_legend=legend(hc(flipud(hc_show_unique)),'Location',x.legend_location,'FontSize',x.legend_font_size);  %flipud since children appear to be added in reverse order
                if ~isempty(x.legend_interpreter)
                    set(h_legend,'Interpreter',x.legend_interpreter);
                end
            end
            %
        end %legend is present
    end
end
%
aux_out.opts_disp=aux.opts_disp;
aux_out.opts_disp_enh=aux.opts_disp_enh;
aux_out.opts_tn2c=aux.opts_tn2c;
return
end

`rs_disp_geofit(gf, aux)`

aux_out=rs_disp_geofit(gf,aux) displays the goodness of fit and model comparison statistics for geometric models fit by rs_geofit

This will create a wirerame plot of goodness of fit (d, see output gfs of rs_geofit for each model, and, optionally, for selected models and comparisons across models that are nested by model type and/or dimension.

Input arguments:

gf (cell 2-D array) –

geometrical fit data, typically gfs{k}.gf, where gfs is the output of rs_geofit, containing the model fits for the kth record of a pair of dataset structures; gf{d_out,d_in} contains the statistics for transforming a dataset of d_in dimensions to one with d_out dimensions
aux (struct) –
auxiliary options, may be omitted, with fields
- opts_dgeo (struct) – options for display, with fields
- opts_check (struct) – options for consistency checking, with field
  - if_warn (int) – 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1

Output arguments:

aux_out ( struct ) –
auxiliary outputs and parameter values used
- warnings (char) – warnings generated in checking for consistency of plotting options
- warn_bad (int) – count of warnings that prevent further processing
- opts_check (struct) – aux.opts_check, with defaults filled in
- opts_dgeo (struct) – aux.opts_dgeo with defaults filled in, and also fields
  - fig_handles (cell array) – handles to the figures created
  - fig_names (cell array) – names of the figures created; no special chars, suitable for use in a file name
  - models_shown (cell array of char) – names of models shown

See also: rs_geofit, PSG_GEOMODELS_FIT, PSG_GEOMODELS_PLOT.

Source code in src/rs_disp_geofit.m

function aux_out=rs_disp_geofit(gf,aux)
% aux_out=rs_disp_geofit(gf,aux) 
% displays the goodness of fit and model comparison statistics for geometric models fit by `rs_geofit`
%
% This will create a wirerame plot of goodness of fit (d, see output gfs of `rs_geofit` for each model,
% and, optionally, for selected models and comparisons across models that are nested by model type and/or dimension.
%
% Args:
%   gf (cell 2-D array): geometrical fit data, typically gfs{k}.gf, where gfs is the output of `rs_geofit`, containing the model fits for the kth record
%      of a pair of `dataset structures`; gf{d_out,d_in} contains the statistics for transforming a dataset of d_in dimensions to one with d_out dimensions
%
%   aux (struct): auxiliary options, may be omitted, with fields
%
%     - opts_dgeo (struct): options for display, with fields
%
%          - **Plot selection**
%          - if_nestbymodel_show (int): 1 to show all nested models, -1 to show only maximally nested models, 0 to omit; default is 1 if shuffles are present and 0 if not
%          - if_nestbydim_show (int): 1 to show nesting by dimension;, 0 to omit; default is 1 if shuffles are present and 0 if not
%          - if_nestbydim_in_show (int): 1 to show nesting by dimension for input, 0 if not; default is if_nestbydim_show
%          - if_nestbydim_out_show (int): 1 to show nesting by dimension for output, 0 if not; defaults is if_nestbydim_show
%          - models_show_select (char or cell array of char): selects the models to be shown in the plot of selected models;
%          a model will be shown if at least one of the strings in models_show_select{:} is present in the model type;
%          e.g., {'_offset','affine'} will select any model whose name contains _offset or affine;
%          default is [], which displays all models. Caution: {} is empty, [] is empty, '' is empty, but {''} and {[]} are not.
%          - if_nestbydim_showd (int): 1 to show d-values for models nested by dimension, 0 to omit; default is 1
%          - if_nestbydim_showd_in (int): 1 to show d-values for models nested by input dimension, 0 to omit; default is if_nestbydim_showd
%          - if_nestbydim_showd_out (int): 1 to show d-values for models nested by output dimension, 0 to omit; default is if_nestbydim_showd
%          - if_diag (int): 0 to plot goodness of fit for all pairs of input and output dimensions, 1 to only plot diagonal values for input dimension equal to ouptut dimension;
%          default is 1 if only on-diagonal values are present, otherwise 0
%
%          - **Plot components**
%          - sig_level (float): significance level for significance flags; default is 0.05
%          - if_showsig (int): 1 to show significance flags based on original denominator, 2 for shuffled denomiator, 3 for both, 0 to omit; default is 3
%          - if_showquant (int): 1 to show quantile of shuffles for nesting at significance level sig_level, 0 to omit; default is 0
%
%          - **Labels and formatting**
%          - out_label (char): label for first  coordinate of gf{}, default is 'output dim'
%          - in_label (char): label for second coordinate of gf{}, defaulti is 'input dim'
%          - dia_label (char): label for both coordinates of gf{}, when diagonal is plotted, default is 'dim'
%          - colors_models (cell array of color specifiers): sequence of colors to use for model wireframes, used in cyclic order;
%           default is {'k','b','c','m','r',[1 0.5 0],[0.7 0.7 0],'g',[.5 .5 .5],[.5 0 0]}; elements can be any valid color specifier
%          - sig_symbols (cell array of char): symbols to mark significant values, sig_symbols{1} for original denominator, sig_symbols{2} for shuffle denominator, default is {'+','x'}
%          - sig_symsize (int): symbol size for significance markers, default is 14
%          - lw_model (int): line width for model, default is 2
%          - lw_nest (int): line width for nested model, default is 2
%          - lw_quant (int): line width for quantiles of shuffles of nested models, default is 1
%          - view (int or float 1-D array): 3-D view descriptor, 2 is 2-d view; can also be azimuth-elevation pair, standard 3-d view is [-37.5 30]; default is 3 unless if_diag=1, in which case default is [0 0], which shows the plot in a plane
% 
%     - opts_check (struct): options for consistency checking, with field
%
%          - if_warn (int): 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
%
% Returns:
%   aux_out (struct): auxiliary outputs and parameter values used
%
%     - warnings (char): warnings generated in checking for consistency of plotting options
%     - warn_bad (int): count of warnings that prevent further processing
%     - opts_check (struct): aux.opts_check, with defaults filled in
%     - opts_dgeo (struct): aux.opts_dgeo with defaults filled in, and also fields
%
%         - fig_handles (cell array):  handles to the figures created
%         - fig_names (cell array): names of the figures created; no special chars, suitable for use in a file name
%         - models_shown (cell array of char): names of models shown
%
%  See also: RS_GEOFIT, PSG_GEOMODELS_FIT, PSG_GEOMODELS_PLOT.
%
if (nargin<=1)
    aux=struct;
end
%
%set up sub-structure options
%
aux=filldefault(aux,'opts_dgeo',struct); %options for this module
%
%plot format options
%
aux.opts_dgeo=filldefault(aux.opts_dgeo,'models_show_select',[]); %strings to select models to show
aux.opts_dgeo=filldefault(aux.opts_dgeo,'sig_level',0.05);
aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_showsig',3); % which significance flags to show(0: none, 1: orig, 2: shuff, 3: both','d',[0 3],3);
aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_nestbydim_showd',1); %1 to show nested-by-dim d-values
aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_nestbydim_in_showd',aux.opts_dgeo.if_nestbydim_showd);
aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_nestbydim_out_showd',aux.opts_dgeo.if_nestbydim_showd);
aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_showquant',0); %1 to show quantile at requested significance level (sig_level)
%
aux.opts_dgeo=filldefault(aux.opts_dgeo,'sig_symbols',{'+','x'});
aux.opts_dgeo=filldefault(aux.opts_dgeo,'sig_symsize',14);
aux.opts_dgeo=filldefault(aux.opts_dgeo,'quant_lines',{'--','-.'});
aux.opts_dgeo=filldefault(aux.opts_dgeo,'colors_mn',{'k','b'});
aux.opts_dgeo=filldefault(aux.opts_dgeo,'colors_models',{'k','b','c','m','r',[1 0.5 0],[0.7 0.7 0],'g',[.5 .5 .5],[.5 0 0]});
aux.opts_dgeo=filldefault(aux.opts_dgeo,'lw_model',2); %line width for a model
aux.opts_dgeo=filldefault(aux.opts_dgeo,'lw_nest',2); %line width for a nested model
aux.opts_dgeo=filldefault(aux.opts_dgeo,'lw_quant',1); %line width for quantiles
aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_omnicolors',1); %1 to use colors from omnibus plots in comparison plots
aux.opts_dgeo=filldefault(aux.opts_dgeo,'out_label','output dim');
aux.opts_dgeo=filldefault(aux.opts_dgeo,'in_label','input dim');
aux.opts_dgeo=filldefault(aux.opts_dgeo,'dia_label','dim');
%
aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_log',0);
%
aux=filldefault(aux,'opts_check',struct); %options for other modules called
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
aux=rs_aux_customize(aux,'rs_disp_geofit');
%
data_out=struct;
aux_out=struct;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
%consistency checks
%
have_data=zeros(size(gf));
gr_fill=struct;
dim_pairs=zeros(0,2);
for ref_dim=1:size(gf,1)
    for adj_dim=1:size(gf,2);
        have_data(ref_dim,adj_dim)=isfield(gf{ref_dim,adj_dim},'model_types_def');
        if have_data(ref_dim,adj_dim)
            dim_pairs(end+1,:)=[ref_dim,adj_dim];
        end
    end
end
ref_dim_list=find(any(have_data,2)'>0);
adj_dim_list=find(any(have_data,1)>0);
npairs=size(dim_pairs,1);
if npairs==0
    wmsg=sprintf('no model fits are present');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1)); %to force a warning output; second argument is a 1 if a bad warning
    disp('cannot proceed');
    disp(aux_out.warnings);
    return;
end
if_onlydiag=double(sum(have_data(:))==sum(diag(have_data)));
if if_onlydiag
    aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_diag',1);
else
    aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_diag',0);
end
if if_onlydiag & (aux.opts_dgeo.if_diag==0)
    wmsg=sprintf('only data for input dim = output dim are available, but 2D plot requested (if_diag=0)');
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn)); %to accumulate warnings and log based on aux_out
end
if aux.opts_dgeo.if_diag==1
    aux.opts_dgeo=filldefault(aux.opts_dgeo,'view',[0 0]);
else
    aux.opts_dgeo=filldefault(aux.opts_dgeo,'view',3);
end

gf_example=gf{dim_pairs(1,1),dim_pairs(1,2)};
nshuffs=size(gf_example.d_shuff,2);
%
%check that requests for nesting display are consistent with presence of statistics
if nshuffs==0
    aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_nestbymodel_show',[]); %1 for all comparisons, -1 for maximal, 0 for none, set to 1 below if shuffles are present
    ask_out=0;
    if ~isempty(aux.opts_dgeo.if_nestbymodel_show)
        ask_out=double(aux.opts_dgeo.if_nestbymodel_show~=0);
    end
    if ask_out
        wmsg=sprintf('no shuffes are present, but display of nesting by models requested; ignored');
        aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn)); %to accumulate warnings and log based on aux_out
    end
    aux.opts_dgeo.if_nestbymodel_show=0;
    %
    aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_nestbydim_show',[]); %1 to show; default to 1 set below if shuffles are present
    aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_nestbydim_in_show',[]);
    aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_nestbydim_out_show',[]);
    if ~isempty(aux.opts_dgeo.if_nestbydim_show)
        ask_nbd=double(aux.opts_dgeo.if_nestbydim_show~=0);
    else
        ask_nbd=0;
    end
    if ~isempty(aux.opts_dgeo.if_nestbydim_in_show)
        ask_nbd_in=double(aux.opts_dgeo.if_nestbydim_in_show~=0);
    else
        ask_nbd_in=0;
    end
    if ~isempty(aux.opts_dgeo.if_nestbydim_out_show)
        ask_nbd_out=double(aux.opts_dgeo.if_nestbydim_out_show~=0);
    else
        ask_nbd_out=0;
    end
    if ask_nbd | ask_nbd_in | ask_nbd_out
        wmsg=sprintf('no shuffes are present, but display of nesting by dimension requested; ignored');
        aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn)); %to accumulate warnings and log based on aux_out
    end
    aux.opts_dgeo.if_nestbydim_show=0;
    aux.opts_dgeo.if_nestbydim_in_show=0;
    aux.opts_dgeo.if_nestbydim_out_show=0;
else
    aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_nestbymodel_show',1); %1 for all comparisons, -1 for only maximal, 0 for none
    aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_nestbydim_show',1); %1 to show
    aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_nestbydim_in_show',aux.opts_dgeo.if_nestbydim_show);
    aux.opts_dgeo=filldefault(aux.opts_dgeo,'if_nestbydim_out_show',aux.opts_dgeo.if_nestbydim_show);
end
%
%create the plots
%
if aux_out.warn_bad>0
    disp('cannot proceed');
    disp(aux_out.warnings);
else
    aux.opts_dgeo.ref_label=aux.opts_dgeo.out_label;
    aux.opts_dgeo.adj_label=aux.opts_dgeo.in_label;
    aux.opts_dgeo=psg_geomodels_plot(gf,aux.opts_dgeo);
    aux.opts_dgeo=rmfield(aux.opts_dgeo,'ref_label');
    aux.opts_dgeo=rmfield(aux.opts_dgeo,'adj_label');
    %
    aux_out.opts_check=aux.opts_check;
    %
    aux_out.opts_dgeo=aux.opts_dgeo;
end
%
return
end

`rs_extract_coordsets(data_in, extract_list, aux)`

[data_out,aux_out]=rs_extract_coordsets(data_in,extract_list,aux) extracts or permutes the records in a dataset structure

Input arguments:

data_in (struct) –
dataset structure containing n records, with fields
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record
extract_list (int 1-D array) –

a subset of [1:n] to extract, the records to extract
aux (struct) –
auxiliary options, may be omitted, with field
- opts_extract (struct) – options for consistency checking, with field
- if_warn (int) – 1 to show warnings, 0 to suppress; default is 1

Output arguments:

data_out ( struct ) –

extracted dataset structure with length(extract_list) records, same format as as data_in1
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_extract (struct) – aux.opts_extract, with defaults filled in

Notes

aux may be omitted; defaults are filled in.
data_out.sets{k}.pipeline is copied from data_in and is not updated.

See also

rs_concat_coordsets.

Source code in src/rs_extract_coordsets.m

function [data_out,aux_out]=rs_extract_coordsets(data_in,extract_list,aux)
% [data_out,aux_out]=rs_extract_coordsets(data_in,extract_list,aux)
% extracts or permutes the records in a `dataset structure`
%
% Args:
%   data_in (struct): `dataset structure` containing n records, with fields
%
%     - ds (cell array): `coordinate structure`, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
%     - sas (cell array): `stimulus metadata structure`, sas{k} is the stimulus metadata for the kth record
%     - sets (cell array): `set metadata structure`, sets{k} is the response metadata for the kth record
%
%   extract_list (int 1-D array): a subset of [1:n] to extract, the records to extract
%
%   aux (struct): auxiliary options, may be omitted, with field
%
%     - opts_extract (struct): options for consistency checking, with field
%
%       - if_warn (int): 1 to show warnings, 0 to suppress; default is 1
% 
% Returns:
%   data_out (struct): extracted `dataset structure` with length(extract_list) records, same format as  as `data_in1`
%
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_extract (struct): aux.opts_extract, with defaults filled in
%
% Note: Notes
%   -  aux may be omitted; defaults are filled in.
%   -  data_out.sets{k}.pipeline is copied from data_in and is not updated.
%
% See also:
%   RS_CONCAT_COORDSETS.
%
if (nargin<=2)
    aux=struct;
end
%set up sub-structure options
%
aux=filldefault(aux,'opts_extract',struct); 
aux.opts_extract=filldefault(aux.opts_extract,'if_warn',1);
%
data_out=struct;
aux_out=struct;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
fns={'ds','sas','sets'}; %fields to extract
%
ns=zeros(1,length(fns)); %check that fields to extract have same length
for ifn=1:length(fns)
    fn=fns{ifn};
    ns(ifn)=length(data_in.(fn));
end
if min(ns)~=max(ns)
    wmsg='input structures have inconsistent lengths';
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_extract.if_warn));
end
if aux_out.warn_bad==0
    extract_ok=ismember(extract_list,[1:max(ns)]);
    if any (extract_ok==0)
        wmsg=sprintf('extract_list has values not in [1:%2.0f]; will be ignored',max(ns));
        aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_extract.if_warn));
    end
    extract_use=extract_list(extract_ok==1);
    for ifn=1:length(fns)
        fn=fns{ifn};
        data_out.(fn)=cell(1,length(extract_use));
        for k=1:length(extract_use)
            data_out.(fn){k}=data_in.(fn){extract_use(k)};            
        end
    end
else
    disp('cannot proceed');
    disp(aux_out.warnings);
end
%    
aux_out.opts_extract=aux.opts_extract;
return
end

`rs_findrays(sa, label, opts_rays)`

[rays,wmsg,opts_rays_used]=rs_findrays(sa,label,opts_rays) creates a ray structure from a single record of a stimulus metadata structure. The ray structure lists the stimuli that lie on rays (points on approximate straight lines), rings (points in a plane at approximately equal distance from the origin), and nearest neighbors, and is used by rs_disp_enh_coordsets.

Input arguments:

sa (struct) –

one record of a stimulus metadata structure, e.g., data.sas{k}, where data is a dataset structure
label (char) –

a paradigm name, allows for non-standard parameters for identifying rays and rings; typically omitted, defaults to []; see note below regarding nondefault parameters
opts_rays (struct) –
options for determining criteria for rays and rings, typically omitted, with fields
- coord_names (cell array of char) – fields of 'sa' to be searched for coordinates in order of priority; default is {'type_coords','btc_specoords','btc_augcoords'}
- ray_permute_raynums (int 1-D array) – permutation to reorder the numbering of rays; default is [] (no permutation)
- ray_tol (float) – tolerance for collinearity of rays; default is 10^-5
- ray_minpts (int) – minimum number of points (other than the origin) to constitute a ray; default is 3
- ray_dirkeep (char) – which rays to keep, default is 'all', alternatives are 'card' (keep only cardinal directions), 'diag' (keep only diagonal directions), 'card_diag' (keep ony cardinal and diagonal directions)
- ray_reorder_ring (int) – 1 to standardize the order of the rings, 0 to omit; default is 1
- ray_plane_jit (float) – small nonzero value to standardize flattening of rays into a plane, 0 to omit; default is 10^-3
- ray_res_ring (float) – tolerance for equal radii for a ring; default is 10^-2
- ray_min_ring (int) – minimum number of points in a ring; default is 4
- ray_mindist_tol (float) – tolerance for ties for nearest-neighbor distance; default is 10^-2

Output arguments:

rays ( struct ) –
ray structure, with fields
- nrays (int) – number of rays
- whichray (int 1-D array) – whichray(istim), in [0 nrays], is the index number of the ray that contains stimulus istim, 0 if not on a ray
- mult (float 1-D array) – mult(istim), in [0 1], is the fractional distance of stimulus istim along its ray, 0 if not on a ray
- endpt (float 2-D array) – endpt(iray,:), with iray in [1 nrays], are the coordinatates of the stimulus at the end of the corresponding ray
- nrings (int) – number of rings
- rings (cell array of struct) – cell array of length nrings; rings{iring}.mult_val is the distance of the ring from the origin, and rings{iring}.coord_ptrs is int 1-d array of indices of stimuli in the ring
- npairs (int) – number of nearest-neighbor pairs
- pairs (int 2-d array) – pairs(ipair,:), with ipair in [1 npairs], are indices of stimuli in a nearest-neighbor pair
wmsg ( char ) –

warning message, if any
opts_rays_used ( struct ) –

opts_rays, with defaults and overrides filled in

Note regarding nondefault ray parameters

For specific paradigm names (passed via 'label'), selected default parameters in opts_rays are overriden as follows.
- 'bcpm24': ray_dirkeep='card_diag' and ray_minpts=2
- 'bcpp55','bcmp55','bcpm55', and 'bcmm55': ray_dirkeep='card'
Additional overrides can be added by editing 'psg_findray_setopts'

See also: rs_disp_enh_coordsets, PSG_DEFOPTS, PSG_FINDRAYS, PSG_TYPE_COORDS_DEF, PSG_TYPE_COORD_UTIL.

Source code in src/rs_findrays.m

function [rays,wmsg,opts_rays_used]=rs_findrays(sa,label,opts_rays)
% [rays,wmsg,opts_rays_used]=rs_findrays(sa,label,opts_rays) creates a `ray structure`
% from a single record of a `stimulus metadata structure`.  The `ray structure` lists the stimuli that lie on rays (points on approximate straight lines),
% rings (points in a plane at approximately equal distance from the origin), and nearest neighbors, and is used by `rs_disp_enh_coordsets`.
%
% Args:
%  sa (struct): one record of a `stimulus metadata structure`, e.g., data.sas{k}, where data is a `dataset structure`
%
%  label (char): a paradigm name, allows for non-standard parameters for identifying rays and rings; typically omitted, defaults to []; see note below regarding nondefault parameters
%
%  opts_rays (struct): options for determining criteria for rays and rings, typically omitted, with fields
%
%     - coord_names (cell array of char): fields of 'sa' to be searched for coordinates in order of priority; default is {'type_coords','btc_specoords','btc_augcoords'}
%     - ray_permute_raynums (int 1-D array): permutation to reorder the numbering of rays; default is [] (no permutation)
%     - ray_tol (float): tolerance for collinearity of rays; default is 10<sup>-5</sup>
%     - ray_minpts (int): minimum number of points (other than the origin) to constitute a ray; default is 3
%     - ray_dirkeep (char): which rays to keep, default is 'all', alternatives are 'card' (keep only cardinal directions), 'diag' (keep only diagonal directions), 'card_diag' (keep ony cardinal and diagonal directions)
%     - ray_reorder_ring (int): 1 to standardize the order of the rings, 0 to omit; default is 1
%     - ray_plane_jit (float): small nonzero value to standardize flattening of rays into a plane, 0 to omit; default is  10<sup>-3</sup>
%     - ray_res_ring (float): tolerance for equal radii for a ring; default is 10<sup>-2</sup>
%     - ray_min_ring (int): minimum number of points in a ring; default is 4
%     - ray_mindist_tol (float): tolerance for ties for nearest-neighbor distance; default is 10<sup>-2</sup>
%
% Returns:
%  rays (struct): `ray structure`, with fields
%
%     - nrays (int): number of rays
%     - whichray (int 1-D array): whichray(istim), in [0 nrays], is the index number of the ray that contains stimulus istim, 0 if not on a ray
%     - mult (float 1-D array): mult(istim), in [0 1], is the fractional distance of stimulus istim along its ray, 0 if not on a ray
%     - endpt (float 2-D array): endpt(iray,:), with iray in [1 nrays], are the coordinatates of the stimulus at the end of the corresponding ray
%     - nrings (int): number of rings
%     - rings (cell array of struct): cell array of length nrings; rings{iring}.mult_val is the distance of the ring from the origin, and rings{iring}.coord_ptrs is int 1-d array of indices of stimuli in the ring
%     - npairs (int): number of nearest-neighbor pairs
%     - pairs (int 2-d array): pairs(ipair,:), with ipair in [1 npairs], are indices of stimuli in a nearest-neighbor pair
%
%  wmsg (char): warning message, if any
%
%  opts_rays_used (struct): opts_rays, with defaults and overrides filled in
%
% Note: Note regarding nondefault ray parameters
%     - For specific paradigm names (passed via 'label'), selected default parameters in opts_rays are overriden as follows.
%
%         - 'bcpm24': ray_dirkeep='card_diag' and ray_minpts=2
%         - 'bcpp55','bcmp55','bcpm55', and 'bcmm55': ray_dirkeep='card'
%
%     - Additional overrides can be added by editing 'psg_findray_setopts'
%
%   See also:  RS_DISP_ENH_COORDSETS, PSG_DEFOPTS, PSG_FINDRAYS, PSG_TYPE_COORDS_DEF, PSG_TYPE_COORD_UTIL.
%

if (nargin<=1)
    label=[];
end
if (nargin<=2)
    opts_rays=struct;
end
opts_rays=getfield(rs_aux_customize(setfield(struct(),'opts_rays',opts_rays),'rs_findrays'),'opts_rays'); %get defaults for ray_reorder_ring and ray_plane_jit
%
opts_rays=filldefault(opts_rays,'coord_names',getfield(psg_defopts,'coord_fields')); % was {'type_coords','btc_specoords','btc_augcoords'});
wmsg=[];
rays=struct;
opts_rays_used=struct; %in case psg_findrays is not called
[stim_coords,coord_names]=psg_type_coords_util(sa,opts_rays.coord_names);
if ~isempty(stim_coords)
    opts_rays=setfield(opts_rays,'coord_names',{coord_names});
    if ~isempty(label)
        opts_rays=psg_findray_setopts(label,opts_rays);
    end
    [rays,opts_rays_used]=psg_findrays(stim_coords,opts_rays);
else
    wmsg=sprintf('cannot find stimulus coordinates, so cannot identify rays');
end
return
end

`rs_geofit(data_in, data_out, aux)`

[gfs,xs,aux_out]=rs_geofit(data_in,data_out,aux) fits geometrical models to the relationship between coordinates in two dataset structures and provides transformation structures that contain these models

Input arguments:

data_in (struct) –
dataset structure that is the starting point of the transformations, with fields
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record
data_out (struct) –

dataset structure that is the target of the transformations, same format as as data_in; number of stimuli must be the same as data_in; stimulus names in data_in.sas{k}.typenames and data_out.sas{k}.typenames need not match but a warning is issued if they do not
aux (struct) –
auxiliary options, may be omitted, with fields
- opts_geof (struct) – specification of transformations to find, with fields
- opts_check (struct) – options for consistency checking, with field
  - if_warn (int) – 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1

Output arguments:

gfs ( cell array of struct ) –
gfs{k}.gf{dim_out,dim_in} contains results for the transformations from record k of data_in to record k of data_out, for the coordinates data_in.ds{k}{dim_in} to data_out.ds{k}{dim_out}. If there is no fitting requested for this dimension pair (see 'dimpairs_method' above), then gfs{k}.gf{dim_out,dim_in} will be empty. It contains the following fields (note that fields with 'shuff' will be absent if if_nestbymodel=0, and fields with 'shuff_nestdim_in' and 'shuff_nestdim_out' will be absent if if_nestbydim_in or if_nestbydim_out are absent)
- model_types_def (struct) – model_types_def.model_types is a cell array of the models fitted; model_types_def.(model).nested is a cell array of the names of the nested models tested
- ref_dim (int) – dimension of the input dataset (same as dim_out)
- adj_dim (int) – dimension of output dataset (same as dim_in)
- opts_geofit (struct) – options used for fitting
- d (float 1-D array) – d(m) is the normlizded error for model m, equal to the sum of the squares of the deviations of data_out.ds{k}{dim_out} from the modeled values, normalized by the sum of their distances from their centroid. Here and below, the index m denotes fits for the model type listed in model_types_def.model_types{m};
- transforms (1-D cell array}: transforms{m} are the parameters for the transformation; see transformation structure for details on how these are parameterized
- d_shuff (float 4-D array) – d_shuff(m,shuff,nest,normtype) is the normalized error for each shuffle of the nested model; normtype=1 normalizes by the centroid of the shuffled data, normtype=2 normalizes by the centroid of the original data
- surrogate_count (int 3-D array) – surrogate_count(m,nest,normtype) counts the number of shuffles for which d_shuff(m,shuff,nest,normtype) is less than d(m)
- nestdim_in_list (int) – list of the lower dimensions used for nesting by input dimension
- d_shuff_nestdim_in (float 4-D array) – d_shuff_nestbydim_in(m,shuff,nest,normtype) is the normalized error for each shuffle for a model with fewer input dimensions; normtype=1 normalizes by the centroid of the shuffled data, normtype=2 normalizes by the centroid of the original data
- surrogate_count_nestdim_in (int 3-D array) – surrogate_count_nestdim_in(m,nest,normtype) counts the number of shuffles for which d_shuff_nestdim_in(m,shuff,nest,normtype) is less than d(m)
- nestdim_out_list (int) – list of the lower dimensions used for nesting by output dimension
- d_shuff_nestdim_out (float 4-D array) – d_shuff_nestbydim_out(m,shuff,nest,normtype) is the normalized error for each shuffle for a model with fewer output dimensions; normtype=1 normalizes by the centroid of the shuffled data, normtype=2 normalizes by the centroid of the original data
- surrogate_count_nestdim_out (int 3-D array) – surrogate_count_nestdim_out(m,nest,normtype) counts the number of shuffles for which d_shuff_nestdim_out(m,shuff,nest,normtype) is less than d(m)
xs ( struct ) –
the transformations fit for dim_out=dim_in, as transformation structures compatible with rs_xform_apply. There is a field xs.(model_name) for each model_name specified in model_list. xs.(model_name), has fields
- class (char) – the transformation class ('mean','procrustes','affine', 'projective','pwaffine')
- xforms (struct) – xs.(model_name).xforms.ts{k}{dim_in}: the transformation to be applied to coordinates in data_in.ds{k}{dim_in} to fit coordinates in data_out.ds{k}{dim_in}. See transformation structure for further details. If no fitting is requested for this dimension pair, then it will be empty.
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_geof (struct) – aux.opts_geof, with defaults filled in
- opts_check (struct) – aux.opts_check, with defaults filled in

Note regarding geometric models

Model types to be fit are specified by the entries in opts_geof.model_list. The following model types are available:
- 'mean': all input values mapped to a single output value
- 'procrustes_noscale_nooffset': rotation (and possibly reflection), no rescaling, no translation
- 'procrustes_scale_nooffset': rotation (and possibly reflection), rescaling allowed, no translation
- 'procrustes_noscale_offset': rotation (and possibly reflection), no rescaling, translation allowed
- 'procrustes_scale_offset': rotation (and possibly reflection), rescaling allowed, translation allowed
- 'affine_nooffset' : linear transformation, no translation
- 'affine_offset': linear transformation, translation allowed
- 'projective': projective transformation
- 'pwaffine': piecewise affine with one cutplane; two linear transformations with agreement on the cut
- 'pwaffine_2': piecewise affine with two cutplanes; four linear transformations, with agreement on the cuts
The list of available model types can be obtained by getfield(psg_geomodels_define,'model_types')
To determine the model class (see transformation structures) for model type mt: m=psg_models_define; getfield(m.(mt),'class')
To determine the models nested in model type mt: m=psg_models_define; getfield(m.(mt),'nested')
See transformation structures for further details on how the models are parameterized and nesting

Note regarding customization

The default model list can be changed by editing the line containing generic.opts_geof.model_list_default in rs_aux_defaults_define, running it once, and saving the workspace as rs_aux_defaults.mat.

Note regarding model definition structure

mdef=rs_geofit() returns a model definition structure, which defines the available models and their characteristics.
mdef.model_types is a cell array {model_name1,model_name2,...} of the names of available models
mdef.(model_name) defines each model
mdef.(model_name).class is the model class: 'mean','procrusetes,'affine','projective','pwaffine' (see rs_xform_apply ??how to hyperlink)
mdef.(model_name).nested lists the names of the nested models

Note regarding nesting and model selection

Nesting by model type: Some models are extensions of others. For example, the affine_offset model extends the affine_noofset model, by allowing offsets. The more general model will always provide a fit that is at least as good as the less-general model, but will have more parameters. The if_nestbymodel option provides a way to determine whether the improvement in fit is better than would be expected by chance.
- To do this, rs_geofit (i) fits with the less-general model, then (ii) shuffles the residuals (the difference between the predicted coordinates in ds_out and the actual coordinates) among the stimuli, and (iii) refits with the more general model. If the more general model provides a fit to the original data that is better than chance, this should rarely result in an improved fit.
- Goodness of fit values and tallies are provided in gfs{k}{dim_out,dim_in}.d_shuff and gfs{k}{dim_out,dim_in}.surrogate_count
- Fine-tuning: if_nestbymodel=+1 vs -1
  - With if_nestbymodel=+1, all nested models specified by model_list are examined
  - with if_nestbymodel=-1, only the maximally nested models are examined. (Model B is considered to be maximally nested in model A if there is no intermediate nested model, i.e., no model X for which B is nested in X, and X is nested in A.)
Nesting by dimension: Models with more coordinates have a greater number of parameters than models with fewer coordinates, and thus may also be viewed as extensions. The if_nestbydim_in and if_nestbydim_out options provide a way to determine whether an improvement due to adding dimensions is better than would be expected by chance.
- To do this, rs_geofit (i) fits a model with fewer coordinates, then (ii) shuffles the added coordinates among the stimuli, and (iii) refits with the more general model. If the more general model provides a fit to the original data that is better than chance, this should rarely result in an improved fit.
- Goodness of fit values and tallies are provided in gfs{k}{dim_out,dim_in}.d_shuff_nestdim_[in|out] and gfs{k}{dim_out,dim_in}.surrogate_count_nestdim_[in|out].
- Fine-tuning: if_nestbydim_[in|out]=+1 vs -1
  - With if_nestbydim_[in|out]=+1, input and output coordinates are used as is for the nesting calculations. This is appropriate if the input and output coordinates themselves are nested, i.e., if data_in.ds{k}{dim_in}(:,1:r)=data_in.ds{k}{r}, for dim_in>r (and similarly for dim_out)
  - However, if this is not the case (for example, if each each set of coordinates data_in.ds{k}{dim_in} is arbitrarily rotated), this analysis could be misleading, since a coordinate in a low-dimensional coordinate set data_in.ds{k}{r} could be rotated into a higher coordinate in data_in.ds{k}(idim}, for idim>r.
  - To avoid this possible pitfall, use if_nestbydim_[in|out]=-1. This applies principal components analysis to the coordinates prior to the analysis of nesting.

See also: rs_aux_customize, rs_check_coordsets, PSG_GEOMODELS_FIT, PSG_GEOMODELS_PLOT, PSG_GEOMODELS_DEFINE, PSG_GEOMODELS_NESTORDER, rs_disp_geofit.

Source code in src/rs_geofit.m

function [gfs,xs,aux_out]=rs_geofit(data_in,data_out,aux)
% [gfs,xs,aux_out]=rs_geofit(data_in,data_out,aux)
% fits geometrical models to the relationship between coordinates in two
% `dataset structures` and provides `transformation structures` that contain these models
%
% Args:
%   data_in (struct): `dataset structure` that is the starting point of the transformations, with fields
%
%     - ds (cell array): `coordinate structure`, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
%     - sas (cell array): `stimulus metadata structure`, sas{k} is the stimulus metadata for the kth record
%     - sets (cell array): `set metadata structure`, sets{k} is the response metadata for the kth record
%
%   data_out (struct): `dataset structure` that is the target of the transformations, same format as  as `data_in`;
%     number of stimuli must be the same as data_in;
%     stimulus names in data_in.sas{k}.typenames and data_out.sas{k}.typenames need not match but a warning is issued if they do not
%
%   aux (struct): auxiliary options, may be omitted, with fields
%
%     - opts_geof (struct): specification of transformations to find, with fields
%
%          - **Model selection**
%          - model_list (char or cell array of char): model types to be fitted; default is values given in `model_list_default`; if [],
%          then requested interactively; see notes below regarding geometric models and model definition structure
%          - model_list_default (char or cell array of char): model types to be fitted when 'model_list' is not specified; default is {'procrustes_scale_offset','affine_offset','projective'};
%            see note below regarding customization
%          - if_center (int): 1 to center the data, i.e., subtract the mean across stimuli from `data_in` and `data_out` before fitting models, 0 to omit; default is 1;  note that if if_center=1, the transformations returned in `gfs` and `xs` apply to the centered data.
%
%          - **Dimension selection**
%          - dim_max_in (int):  maximum dimension of input dataset to use, defaults to 10
%          - dim_max_out (int): maximum dimension of output dataset to use, defaults to `dim_max_in`
%          - dimpairs_method (char): specifies pairing of dimensions between `data_in` and `data_out`, default is 'equal'
%
%              - 'equal': input dimension = output dimension
%              - 'all': all pairings of dimensions available in `data_in` up to `dim_max_in`, with all and dimensions available i `data_out`, up to `dim_max_out`
%              - 'din_lteq_dout': as in 'all', but input dimension must be less than or equal to output dimension
%              - 'din_gteq_dout': as in 'all', but input dimension must be greater than or equal to output dimension
%              - 'list': the pairings specified by aux.opts_geof.dimpairs_list
%
%          - dimpairs_list (int 2-D array): two-column array of pairs of dimensions for input and output, default is repmat([1:dim_max_in]',[1 2])
%
%          - **Statistics**
%          - if_stats (int): 1 to enable statistics, 0 to omit; default is 1; a value of 0 will override a nonzero `if_nestbymodel` and `if_nestbydim`
%          - nshuffs (int): number of shuffles for `if_nestbymodel` and `if_nestbydim`; default is 100 if if_stats=1, 0 if if_stats=0
%          - if_frozen (int): random number control for shuffles and initialization; 1 for same numbers every run, 0 for different random numbers each run, negative integer for a fixed seed each run; default is 1
%          - if_nestbymodel (int): 1 to do statistics on nesting by model, 0 to omit, -1 to only do statistics for maximally nested models; default is 1; see note below regarding nesting
%          - if_nestbydim (int): +/-1 to do statistics for nesting by dimension, 0 to omit; default is 0; see note below regarding nesting
%          - if_nestbydim_in (int): +/-1 to do statistics for nesting by dimension of input, 0 to omit; default is if_nestbydim; see note below regarding nesting
%          - if_nestbydim_out (int): +/-1 to do statistics for nesting by dimension of output, 0 to omit; default is if_nestbydim; see note below regarding nesting
%          - if_keep_transforms (int): 1 to keep transforms from shuffles, 0 to omit; default is 0
%          - if_keep_transforms_nestbymodel (int): 1 to keep transforms from nesting by model, 0 to omit; default is 0 if if_nestbymodel=0, 1 otherwise
%          - if_keep_transforms_nestbydim_in (int): 1 to keep transforms from nesting by dimension of input, 0 to omit; default is 0 if if_nestbydim_in=0, 1 otherwise
%          - if_keep_transforms_nestbydim_out (int): 1 to keep transforms from nesting by dimension of output, 0 to omit; default is 0 if if_nestbydim_out=0, 1 otherwise
%
%          - **Logging and optimization**
%          - if_log (int): 1 to log overall progress, 0 to omit; default is 1
%          - if_fit_summary (int): 1 to log a summary of fits, 0 to omit; default is 1
%          - if_fit_log (int): 1 for a detailed log of fitting, 0 to omit; default is 0
%          - if_warn (int): 1 to show warnings, 0 to omit; default is 1
%          - persp_method (char): method for finding projective transformations, options are 'fmin','oneshot', or 'best'; default is 'best'
%
%              - 'fmin': uses an iterative method to search for the denominator; for each trial denominator, numerator parameters are determined by standard least-squares
%              - 'oneshot': uses method 2 of Zhengyou Zhang, Microsoft Research Technical Report MSR-TR-2010-63 (1993, revised 2010); well-suited if the projective transformation is a close fit
%              - 'best' uses both methods and chooses the best-fit
%
%     - opts_check (struct): options for consistency checking, with field
%
%          - if_warn (int): 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
%
% Returns:
%   gfs (cell array of struct): gfs{k}.gf{dim_out,dim_in} contains results for the transformations from record k of `data_in` to record k of `data_out`,
%     for the coordinates data_in.ds{k}{dim_in} to data_out.ds{k}{dim_out}. If there is no fitting requested for this dimension pair (see 'dimpairs_method' above), then gfs{k}.gf{dim_out,dim_in} will be empty.
%     It contains the following fields (note that fields with 'shuff' will be absent if if_nestbymodel=0, and fields with 'shuff_nestdim_in' 
%     and 'shuff_nestdim_out' will be absent if if_nestbydim_in or if_nestbydim_out are absent)
%
%     - model_types_def (struct): model_types_def.model_types is a cell array of the models fitted; model_types_def.(model).nested is a cell array of the names of the nested models tested
%     - ref_dim (int): dimension of the input dataset (same as dim_out)
%     - adj_dim (int): dimension of output dataset (same as dim_in)
%     - opts_geofit (struct): options used for fitting
%     - d (float 1-D array): d(m) is the normlizded error for model m, equal to the sum of the squares of the deviations of data_out.ds{k}{dim_out} from the modeled values, normalized by the sum of their distances from their centroid.
%     Here and below, the index m denotes fits for the model type listed in model_types_def.model_types{m}; 
%     - transforms (1-D cell array}: transforms{m} are the parameters for the transformation; see `transformation structure` for details on how these are parameterized
%     - d_shuff (float 4-D array): d_shuff(m,shuff,nest,normtype) is the normalized error for each shuffle of the nested model; normtype=1  normalizes by the centroid of the shuffled data, normtype=2 normalizes by the centroid of the original data
%     - surrogate_count (int 3-D array): surrogate_count(m,nest,normtype) counts the number of shuffles for which d_shuff(m,shuff,nest,normtype) is less than d(m)
%     - nestdim_in_list (int): list of the lower dimensions used for nesting by input dimension
%     - d_shuff_nestdim_in (float 4-D array): d_shuff_nestbydim_in(m,shuff,nest,normtype) is the normalized error for each shuffle for a model with fewer input dimensions; normtype=1  normalizes by the centroid of  the shuffled data, normtype=2 normalizes by the centroid of the original data
%     - surrogate_count_nestdim_in (int 3-D array): surrogate_count_nestdim_in(m,nest,normtype) counts the number of shuffles for which d_shuff_nestdim_in(m,shuff,nest,normtype) is less than d(m)
%     - nestdim_out_list (int): list of the lower dimensions used for nesting by output dimension
%     - d_shuff_nestdim_out (float 4-D array): d_shuff_nestbydim_out(m,shuff,nest,normtype) is the normalized error for each shuffle for a model with fewer output dimensions; normtype=1  normalizes by the centroid of the shuffled data, normtype=2 normalizes by the centroid of the original data
%     - surrogate_count_nestdim_out (int 3-D array): surrogate_count_nestdim_out(m,nest,normtype) counts the number of shuffles for which d_shuff_nestdim_out(m,shuff,nest,normtype) is less than d(m)
%
%   xs (struct): the transformations fit for dim_out=dim_in, as `transformation structures` compatible with `rs_xform_apply`. There is a field xs.(model_name) for each model_name specified in model_list. xs.(model_name), has fields
%
%     - class (char): the transformation class ('mean','procrustes','affine', 'projective','pwaffine')
%     - xforms (struct): xs.(model_name).xforms.ts{k}{dim_in}: the transformation to be
%     applied to coordinates in data_in.ds{k}{dim_in} to fit coordinates in
%     data_out.ds{k}{dim_in}. See `transformation structure` for further details. If no fitting is requested for this dimension pair, then it will be empty.
%
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_geof (struct): aux.opts_geof, with defaults filled in
%     - opts_check (struct): aux.opts_check, with defaults filled in
%
% Note: Note regarding geometric models
%    - Model types to be fit are specified by the entries in opts_geof.model_list. The following model types are available:
%
%        - 'mean': all input values mapped to a single output value 
%        - 'procrustes_noscale_nooffset': rotation (and possibly reflection), no rescaling, no translation
%        - 'procrustes_scale_nooffset': rotation (and possibly reflection), rescaling allowed, no translation
%        - 'procrustes_noscale_offset': rotation (and possibly reflection), no rescaling, translation allowed
%        - 'procrustes_scale_offset': rotation (and possibly reflection), rescaling allowed, translation allowed
%        - 'affine_nooffset' : linear transformation, no translation
%        - 'affine_offset': linear transformation, translation allowed
%        - 'projective': projective transformation
%        - 'pwaffine': piecewise affine with one cutplane; two linear transformations with agreement on the cut
%        - 'pwaffine_2': piecewise affine with two cutplanes; four linear transformations, with agreement on the cuts
%
%    - The list of available model types can be obtained by getfield(psg_geomodels_define,'model_types')
%    - To determine the model class (see `transformation structures`) for model type mt: m=psg_models_define; getfield(m.(mt),'class')
%    - To determine the models nested in model type mt:  `m=psg_models_define; getfield(m.(mt),'nested')`
%    - See `transformation structures` for further details on how the models are parameterized and nesting
%
% Note: Note regarding customization
%    The default model list can be changed by editing the line containing generic.opts_geof.model_list_default in `rs_aux_defaults_define`, running it 
%    once, and saving the workspace as rs_aux_defaults.mat.
%   
% Note: Note regarding model definition structure
%    - mdef=rs_geofit() returns a model definition structure, which defines the available models and their characteristics.
%    - mdef.model_types is a cell array {model_name1,model_name2,...} of the names of available models
%    - mdef.(model_name) defines each model
%    - mdef.(model_name).class is the model class: 'mean','procrusetes,'affine','projective','pwaffine' (see `rs_xform_apply` ??how to hyperlink)
%    - mdef.(model_name).nested lists the names of the nested models
%
% Note: Note regarding nesting and model selection
%    - Nesting by model type: Some models are extensions of others. For example, the affine_offset model extends the affine_noofset model, by allowing offsets.
%    The more general model will always provide a fit that is at least as good as the less-general model, but will have more parameters.  The if_nestbymodel option provides a way to determine
%    whether the improvement in fit is better than would be expected by chance.
%
%        - To do this, rs_geofit (i) fits with the less-general model, then (ii) shuffles the residuals (the difference between the predicted
%        coordinates in ds_out and the actual coordinates) among the stimuli, and (iii) refits with the more general model. If the more general
%        model provides a fit to the original data that is better than chance, this should rarely result in an improved fit.
%        - Goodness of fit values and tallies are provided in
%        gfs{k}{dim_out,dim_in}.d_shuff and gfs{k}{dim_out,dim_in}.surrogate_count
%        - Fine-tuning: if_nestbymodel=+1 vs -1
%
%            - With if_nestbymodel=+1, all nested models specified by model_list are examined
%            - with if_nestbymodel=-1, only the maximally nested models are examined. (Model B is considered to be maximally nested in model A if there is no intermediate nested model, i.e., no model X for which B is nested in X, and X is nested in A.)
%    
%    - Nesting by dimension: Models with more coordinates have a greater number of parameters than models with fewer coordinates, and thus may also be viewed as extensions.
%    The if_nestbydim_in and if_nestbydim_out options provide a way to determine whether an improvement due to adding dimensions is better than would be expected by chance.
%   
%        - To do this, rs_geofit (i) fits a model with fewer coordinates, then (ii) shuffles the added coordinates among the stimuli, and (iii) refits with the more general model. If the more general
%        model provides a fit to the original data that is better than chance, this should rarely result in an improved fit.
%        - Goodness of fit values and tallies are provided in gfs{k}{dim_out,dim_in}.d_shuff_nestdim_[in|out] and gfs{k}{dim_out,dim_in}.surrogate_count_nestdim_[in|out].
%        - Fine-tuning: if_nestbydim_[in|out]=+1 vs -1
%
%            - With if_nestbydim_[in|out]=+1, input and output coordinates are
%            used as is for the nesting calculations.  This is appropriate if the input and output coordinates themselves are nested, i.e., if data_in.ds{k}{dim_in}(:,1:r)=data_in.ds{k}{r}, for dim_in>r (and similarly for dim_out)
%            - However, if this is not the case (for example, if each each set of
%            coordinates data_in.ds{k}{dim_in} is arbitrarily rotated), this analysis could be misleading, since a coordinate in a low-dimensional coordinate set data_in.ds{k}{r} could be rotated into a higher coordinate in data_in.ds{k}(idim}, for idim>r.
%            - To avoid this possible pitfall, use if_nestbydim_[in|out]=-1.  This applies  principal components analysis to the coordinates prior to the analysis of nesting.
%
%  See also: RS_AUX_CUSTOMIZE, RS_CHECK_COORDSETS, PSG_GEOMODELS_FIT, PSG_GEOMODELS_PLOT, PSG_GEOMODELS_DEFINE,
%    PSG_GEOMODELS_NESTORDER, RS_DISP_GEOFIT.
%
%
fields_unchanged={'nshuffs','if_nestbydim','if_center','if_frozen','persp_method','if_nestbymodel','if_nestbydim_in','if_nestbydim_out',...
    'if_keep_transforms','if_keep_transforms_nestbymodel','if_keep_transforms_nestbydim_in','if_keep_transforms_nestbydim_out'};
%
psg_geomodels_def=psg_geomodels_define(); %access the available models and their characteristics
%
%special case: display available models
if nargin==0
    gfs=psg_geomodels_def;
    xs=struct;
    aux_out=struct;
    return
end
if (nargin<=2)
    aux=struct;
end
%
%set up sub-structure options
%
aux=filldefault(aux,'opts_geof',struct); %options for this module
aux.opts_geof=filldefault(aux.opts_geof,'dimpairs_method','equal');
aux.opts_geof=filldefault(aux.opts_geof,'dim_max_in',10);
aux.opts_geof=filldefault(aux.opts_geof,'dim_max_out',aux.opts_geof.dim_max_in);
aux.opts_geof=filldefault(aux.opts_geof,'dimpairs_list',repmat([1:aux.opts_geof.dim_max_in]',[1 2]));
aux.opts_geof=filldefault(aux.opts_geof,'if_stats',1);
aux.opts_geof=filldefault(aux.opts_geof,'if_nestbymodel',1);
aux.opts_geof=filldefault(aux.opts_geof,'if_nestbydim',0);
aux.opts_geof=filldefault(aux.opts_geof,'if_nestbydim_in',aux.opts_geof.if_nestbydim);
aux.opts_geof=filldefault(aux.opts_geof,'if_nestbydim_out',aux.opts_geof.if_nestbydim);
aux.opts_geof=filldefault(aux.opts_geof,'if_center',1);
aux.opts_geof=filldefault(aux.opts_geof,'if_frozen',1);
aux.opts_geof=filldefault(aux.opts_geof,'if_fit_summary',1);
aux.opts_geof=filldefault(aux.opts_geof,'if_fit_log',0);
aux.opts_geof=filldefault(aux.opts_geof,'if_warn',1);
aux.opts_geof=filldefault(aux.opts_geof,'if_log',1);
aux.opts_geof=filldefault(aux.opts_geof,'persp_method','best');
%
aux.opts_geof=filldefault(aux.opts_geof,'if_keep_transforms',0);
aux.opts_geof=filldefault(aux.opts_geof,'if_keep_transforms_nestbymodel',aux.opts_geof.if_keep_transforms*abs(aux.opts_geof.if_nestbymodel));
aux.opts_geof=filldefault(aux.opts_geof,'if_keep_transforms_nestbydim_in',aux.opts_geof.if_keep_transforms*abs(aux.opts_geof.if_nestbydim_in));
aux.opts_geof=filldefault(aux.opts_geof,'if_keep_transforms_nestbydim_out',aux.opts_geof.if_keep_transforms*abs(aux.opts_geof.if_nestbydim_out));
%
aux=filldefault(aux,'opts_check',struct); %options for other modules called
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
if aux.opts_geof.if_stats
    aux.opts_geof=filldefault(aux.opts_geof,'nshuffs',100);
else
    aux.opts_geof=filldefault(aux.opts_geof,'nshuffs',0);
    aux.opts_geof.if_nestbymodel=0;
    aux.opts_geof.if_nestbydim=0;
    aux.opts_geof.if_nestbydim_in=0;
    aux.opts_geof.if_nestbydim_out=0;
end
%
aux=rs_aux_customize(aux,'rs_geofit');
%
aux.opts_geof=filldefault(aux.opts_geof,'model_list',aux.opts_geof.model_list_default);
%
aux_out=struct;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
xs=[];
gfs=[];
%
%parse model names and determine order of models to compute
%
[aux,aux_out]=rs_geofit_getmodels(aux,psg_geomodels_def,aux_out);
%aux.opts_geof.model_list is model models in order to be evaluated
%aux.opts_geof.model_definitions is the model definition structure
nmodels=length(aux.opts_geof.model_list);
%
%check consistency within input and output
%
check_in=rs_check_coordsets(data_in,aux.opts_check);
aux_out.warnings=strvcat(aux_out.warnings,check_in.warnings);
check_out=rs_check_coordsets(data_out,aux.opts_check);
aux_out.warnings=strvcat(aux_out.warnings,check_out.warnings);
if aux_out.warn_bad>0
    disp('cannot proceed');
    return
end
dim_list_in=check_in.dim_list_union;
dim_list_out=check_out.dim_list_union;
%
%check consistency between input and output
%
nstims_in=check_in.nstims_each;
nstims_out=check_out.nstims_each;
if nstims_in~=nstims_out
    wmsg=sprintf('mismatch in number of stimuli: input dataset has %3.0f stimuli, output dataset has %3.0f stimuli',nstims_in,nstims_out);
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
end
if length(union(check_in.typenames_union,check_out.typenames_union))~=length(intersect(check_in.typenames_inter,check_out.typenames_inter))
    wmsg=sprintf('data_in and data_out have different typenames');
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
if aux_out.warn_bad>0
    disp('cannot proceed');
    return
end
z=aux.opts_geof;
[dpx,dpy]=meshgrid(1:min(z.dim_max_in,max(dim_list_in)),1:min(z.dim_max_out,max(dim_list_out)));
dimpairs_all=[dpx(:),dpy(:)];
if ((ndims(z.dimpairs_list)~=2) | (size(z.dimpairs_list,2)~=2))
    wmsg=sprintf('dimension pairing list has unexpected shape; default pairing '' equal'' used');
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',z.if_warn));
    z.dimpairs_method='equal';
end
switch z.dimpairs_method %determine coordinate groups
    case 'all'
        z.dimpairs_list=dimpairs_all;
    case 'equal'
        z.dimpairs_list=dimpairs_all(find(dimpairs_all(:,1)==dimpairs_all(:,2)),:);
    case 'din_lteq_dout'
        z.dimpairs_list=dimpairs_all(find(dimpairs_all(:,1)<=dimpairs_all(:,2)),:);
    case 'din_gteq_dout'
        z.dimpairs_list=dimpairs_all(find(dimpairs_all(:,1)>=dimpairs_all(:,2)),:);
    case 'list'
    otherwise
        wmsg=sprintf('dimension pairing method  (%s) not recognized; ''equal'' used',z.dimpairs_method);
        z.dimpairs_method='equal';
        aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',z.if_warn));
end
dp_bad=union(find(any(z.dimpairs_list<1,2)),find(any(z.dimpairs_list~=round(z.dimpairs_list),2)));
if ~isempty(dp_bad)
    wmsg=sprintf('dimension pairing list has non-integer values or values less than 1, ignored');
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',z.if_warn));
    z.dimpairs_list=z.dimpairs_list(setdiff(1:size(z.dimpairs_list,1),dp_bad),:);
end
dp_sel=intersect(find(z.dimpairs_list(:,1)<=z.dim_max_in),find(z.dimpairs_list(:,2)<=z.dim_max_out));
z.dimpairs_list=z.dimpairs_list(dp_sel,:);
%
nsets=max(check_in.nsets,check_out.nsets);
gfs=cell(1,nsets);
xs=struct;
for k=1:nmodels
    model_name=z.model_list{k};
    xs.(model_name)=struct;
    xs.(model_name).class=z.model_definitions.(model_name).class;
    xs.(model_name).xforms=struct;
end    
%
opts_psgfit_base=struct;
opts_psgfit_base.model_types_def=z.model_definitions;
opts_psgfit_base.if_log=z.if_fit_log;
opts_psgfit_base.if_summary=z.if_fit_summary;
%fields passed from aux.opts_geof to opts_psgfit with no change
for k=1:length(fields_unchanged)
    opts_psgfit_base.(fields_unchanged{k})=z.(fields_unchanged{k});
end
%
for iset=1:nsets
    iset_in=1+mod(iset-1,check_in.nsets);
    iset_out=1+mod(iset-1,check_out.nsets);
    d_adj=data_in.ds{iset_in};
    d_ref=data_out.ds{iset_out};
    if z.if_log
        disp(sprintf('geometric model fits: input set %2.0f, output set %2.0f',iset_in,iset_out));
    end
    %
    %do the fits
    %
    opts_psgfit=opts_psgfit_base;
    opts_psgfit.dimpairs_list=z.dimpairs_list;
    opts_psgfit.if_keep_opts_model_used=0; %eliminate some un-needed fields
    [gf,opts_psgfit_used]=psg_geomodels_fit(d_ref,d_adj,opts_psgfit);
    gfs{iset}.gf=gf;
    z.warnings_fit{iset}=opts_psgfit_used.warnings;
    if (z.if_warn & ~isempty(z.warnings_fit{iset}))
        disp('warnings encountered during fits:')
        disp(z.warnings_fit{iset});
    end
    %
    %format the transformations when dim_in=dim_out
    %
    for k=1:nmodels
        model_name=z.model_list{k};
        for idim=1:min(size(gf))
            if ~isempty(gf{idim,idim})
                if isfield(gf{idim,idim},'transforms')
                    xs.(model_name).xforms.ts{iset}{1,idim}=gf{idim,idim}.transforms{k};
                end
            end
        end
    end
%   xs.(model_name).class: the transformation class ('affine', 'projective','pwaffine','pwprojective')
%   xs.(model_name).xforms.ts{k}{idim}: the transformation to be applied to dataset k, coordinate set of dimension idim
%     (this will be empty if there is no transformation in gfs{k}.gf{idim,idim}
end
% 
% return options as used
%
aux_out.opts_check=aux.opts_check;
aux_out.opts_geof=z;
return
end

`rs_get_coordsets(fullnames, aux)`

[data_out,aux_out]=rs_get_coordsets(fullnames,aux): reads one or more coordinate files into a dataset structure

This is the preferred method for bringing coordinates from one or more files into a dataset structure suitable for display and geometrical analysis.

For files generated by analysis of similarity judgments via [??how to refer to Python package], this will also import the goodness-of-fit data.
A ray structure will be created.
If only a single data file is to be read, rs_read_coorddata can also be used.
This can also create a coordinate structure based on a quadratic form model, and can augment binary texture domain data by symmetry.
To import coordinates generated externally or that exist as arrays rather than in a file, use rs_import_coordsets.
For non-interactive reading, provide aux.nsets, aux.opts_read.data_fullnames, aux.opts_read.input_type, and set aux.opts_read.if_auto=1.

Input arguments:

fullnames (char or cell array of char) –

full file name with path; if empty, it will be requested interactively. File names must contain the string '_coords'. See note below regarding setup files.
aux (struct) –
a structure, can be omitted, with fields
- nsets (int) – number of records to read, if zero (default), then requested at console, with console option to switch to gui
- opts_read (struct) – can be omitted, with fields listed below
- if_gui (int) – 1 to use graphical interface to get files if file names are not supplied (default), 0 to use console; ignored if nsets=0; see note below regarding customization
- if_log (int) – 1 to log progress, 0 to omit; default is 1; see note below regarding customization
- if_auto (int) – 1 not to ask for confirmations, 0 to ask; default is 0; see note below regarding customization
- nfiles_max (int) – maximum number of files to read; default is 100
- input_type (int or int 1-D array) – 1 for reading a coordinate file, 2 for reading a quadratic form model, 0 to ask; cycled through for each dataset; default is 0; see note below regarding customization
- data_fullname_def (char) – prompt for data file name; see note below regarding customization
- ui_filter (char) – file name filter that appears in graphical user interface when if_gui=1; defaults to '*_coords';see note below regarding customization
- if_warn (int) – 1 to show warnings, 0 to omit; default is 1; see note below regarding customization

Output arguments:

data_out ( struct ) –
dataset structure, containing one records, with fields
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- rayss (cell array of struct) – rayss{k} is the ray structure for the kth record
- opts_read (cell array of struct) – opts_read{k} is aux.opts_read for reading the kth record, with defaults filled in
- opts_rays (cell array of struct) – opts_rays{k} are the options used for creating the ray structure for the kth record
- opts_qpred (cell array of struct) – opts_qpred{k} are the options used for creating the quadratic form model for the kth record along with
  - var_tot (float) – total variance of the coordinates
  - var_ex (float 1-D array) – variance explained by each dimension of the model
  - predcoords (float 2-D array) – predcoords(istim,idim) is the model coordinate for stimulus istim in dimension idim
- opts_check (struct) – aux.opts_check, with defaults filled in
- syms_list (struct) – structure with results of symmetry augmentation
  - sym_apply (char) – the symmetry applied, see 'sym_apply' above
  - primary (int 1-D array) – primary(k) is pointer to the data that was augmented by symmetry to yield the kth record
  - aug (int 1-D array) – aug(k) indicates the symmetry that was used to yield the kth record, e.g., if there are n_sym total symmetries (e.g., n_sym=8 for 'full'), then aug is in [1:n_sym]. The pairs [primary(k) aug(k)] are unique; a contiguous block of n_sym pairs share one value of primary(k) and have sequential values of aug(k)
  - syms_applied (cell array of char) – syms_applied{k} are the binary texture coordinates after symmetry transformation for record k

Note regarding domain names, stimulus coordinates, and setup files

The domain name is determined from the data file name based on the string preceding '_coords' (or, opts_read.coord_string, if customized). This in turn determines how stimulus coordinates are specified.
- For non-reserved domain names (opts_read.type_class_aux, or any domain in opts_read.domain_list_def), stimulus coordinates are specified by 'type_coords' and no setup file is used.
- Other domain names are reserved, and stimulus coordinates are specified by a 'setup file', whose suggested name is constructed from the data file name up to the string preceding opts_read.coord_string, and appending opts_read.setup_suffix
- The domain name is also used to determine a 'type_class', which is used internally. This is carried out in psg_coorddata_parsename, as follows:
  - The domains 'faces*' are reserved; type_class is faces_mpi; used with a setup file
  - The domain 'irgb' is reserved; type_class is 'irgb'; used with a setup file
  - The domain 'mater' is reserved; type_class is 'mater'; used with a setup file
  - The domain opts_read.type_class_aux is NOT reserved; type class is type_class_aux; no setup file used
  - Any domain that is listed in opts_read.domain_list_def is NOT reserved; type class is 'domain'; no setup file used
- Otherwise, type_class is set to opts_read.type_class_def, ('btc' unless customized); used with a setup file
When a dataset structure is written by rs_write_coordsets, the setup file information, if any, will be embedded. When that file is read in and an embedded setup file is detected, no setup file is requested.
Also see note below regarding customization

Note regarding quadratic form model

Quadratic form models may be used to generate a coordinate structure based on stimulus coordinates.
The file must a cell array r, and each r{imodel} has at least has structure r The quadratic form model file must contain a field r, which must contain
- results.qfit (float 2-D array): a symmetric matrix indicating the quadratic form which, when applied to stimulus coordinates, yields the model coordinates
- setup.label (char): a label for the model
See demos/opposites_qform_example.mat for an example file with four quadratic form models, applicable to a stimulus set in which stimuli have three coordinates

Note regarding customization

The defaults for the following parameters can be set by editing the line containing generic.opts_read.[param_name] in rs_aux_defaults_define, running it once, and saving the workspace as rs_aux_defaults.mat.
- if_gui
- ui_filter
- if_log
- if_warn
- if_auto
- data_fullname_def
- setup_fullname_def
- domain_list_def
- setup_suffix
- permutes_ok
- coord_string
- type_class_aux
- if_symaug
- sym_apply
- if_symaug_log
The defaults for the following parameters can be set by editing the line containing generic.opts_qpred.[param_name] in rs_aux_defaults_define, running it once, and saving the workspace as rs_aux_defaults.mat.
- qform_datafile_def
- qform_modeltype

See also: rs_aux_customize, rs_findrays, rs_check_coordsets, PSG_GET_COORDSETS, PSG_COORDDATA_PARSENAME.

Source code in src/rs_get_coordsets.m

function [data_out,aux_out]=rs_get_coordsets(fullnames,aux)
% [data_out,aux_out]=rs_get_coordsets(fullnames,aux): reads one or more `coordinate files` into a `dataset structure`
% 
% This is the preferred method for bringing coordinates from one or more files into a `dataset structure` suitable for display and geometrical analysis.
%
%  - For files generated by analysis of similarity judgments via [??how to refer to Python package], this will also import the goodness-of-fit data.
%  - A `ray structure` will be created.
%  - If only a single data file is to be read, `rs_read_coorddata` can also be used.
%  - This can also create a `coordinate structure` based on a `quadratic form model`, and can augment `binary texture domain` data by symmetry.
%  - To import coordinates generated externally or that exist as arrays rather than in a file, use `rs_import_coordsets`.
%
%  - For non-interactive reading, provide aux.nsets, aux.opts_read.data_fullnames, aux.opts_read.input_type, and set aux.opts_read.if_auto=1.
%
% Args:
%   fullnames (char or cell array of char): full file name with path; if empty, it will be requested interactively. File names must contain the string '_coords'. See note below regarding setup files.
%
%   aux (struct): a structure, can be omitted, with fields 
%
%     - nsets (int): number of records to read, if zero (default), then requested at console, with console option to switch to gui
%     - opts_read (struct): can be omitted, with fields listed below
%
%        - if_gui (int): 1 to use graphical interface to get files if file names are not supplied (default), 0 to use console; ignored if nsets=0; see note below regarding customization
%        - if_log (int): 1 to log progress, 0 to omit; default is 1; see note below regarding customization
%        - if_auto (int): 1 not to ask for confirmations, 0 to ask; default is 0; see note below regarding customization
%        - nfiles_max (int): maximum number of files to read; default is 100
%        - input_type (int or int 1-D array): 1 for reading a coordinate file, 2 for reading a `quadratic form model`, 0 to ask; cycled through for each dataset; default is 0;  see note below regarding customization
%        - data_fullname_def (char): prompt for data file name; see note below regarding customization
%        - ui_filter (char): file name filter that appears in graphical user interface when if_gui=1; defaults to '*_coords';see note below regarding customization
%        - if_warn (int): 1 to show warnings, 0 to omit;  default is 1; see note below regarding customization
%
%        - **Options related to stimulus coordinates, domains, and setup files (see notes below)** 
%        - type_coords (float 2-D array): stimulus coordinates to use for non-reserved domain names, one row for each stimulus; default is empty
%        - type_coords_def (char): stimulus coordinates to use if type_coords is unspecified or empty; options are 'none', 'zeros', 'ones', or 'eye', see `rs_import_coordsets`; default is 'none'; see `rs_import_coordsets` for further details and how to customize
%        - domain_list_def (cell array): paradigm names for generic domain experiments; default is {'cars','tools','dwellings'}; determined by customization during install
%        - paradigm_type_def (char): default paradigm type; default is determined by customization during install
%        - coord_string (char): a string in a data file name that delineates the end of the domain name; defaults to '_coords'
%        - setup_fullnames (cell array): setup file full names; if empty, will be requested at console or via gui
%        - setup_fullname_def (char): prompt for setup file name
%        - setup_suffix (char): suffix added to paradigm name to create the setup file name; default is 'S'
%        - type_class_aux (char): a specified domain name and type_class
%
%        - **Options for symmetry augmentation (`binary texture domain` data only)**
%        - if_symaug (int): 1 to enable symmetry augmentation, 0 to omit, -1 to ask; default is 0
%        - if_symaug_log (int): 1 to log symmetry augmentation, 0 to omit; default is 0
%        - sym_apply (char): type of symmetry to apply; default is 'full', options are  'hflip','vflip','rot180','rot90','hvflip','diag','full','none'
%
%        - **Options for internal use and maintenance**
%        - permutes (struct): typically omitted; each field is a suggested ray permutation for the corresponding paradigm name, e.g., permutes.bgca=[2 1 3 4] specifies a reordering of the rays for paradigm 'bgca'
%        - permutes_ok (int): typically omitted; 1 to accept suggested ray permutation, 0 to keep standard order; default is 1; see note below regarding customization
%        - if_uselocal (int): typically omitted; 0 to use options in rs_aux_defaults, 1 is reserved for maintenance; default is 0
%
%     - opts_qpred (struct): a structure, can be omitted, with options for `quadratic form model`, with fields
%
%        - qform_datafile_def (char): default name of the file with quadratic form models containing a structure r; see notes below quadratic form model and customization
%        - qform_modeltype (int or int 1-D array): list of indexes into qform_datafile_def for choice of model, used cyclically across records as needed. 0 -> ask at console; detault is 1 see note below re customization
%        - if_log (int): 1 to log calculation of quadratic form model, 0 to omit; default is 0
%        - if_pca_centroid (int): 1 to force centroid of coordinates to 0 via PCA method, 0 to omit; default is 1
%        - negeig_makezero (int): 1 to set negative eigenvalues to 0, 1 to omit; default is 1
%        - negtol (float): eigenvalues between 0 and -negtol are set to zero before checking for negative eigenvalues; default is 10^-7
%
%     - opts_rays (struct): options for identifying rays, may be omitted; see `rs_findrays` for details
%
%     - opts_check (struct): options for consistency checking, with field
%
%         - if_warn (int): 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
%
% Returns:
%   data_out (struct): `dataset structure`, containing one records, with fields
%
%     - ds (cell array): `coordinate structure`, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
%     - sas (cell array): `stimulus metadata structure`, sas{k} is the stimulus metadata for the kth record
%     - sets (cell array): `set metadata structure`, sets{k} is the response metadata for the kth record
%
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - rayss (cell array of struct): rayss{k} is the `ray structure` for the kth record
%     - opts_read (cell array of struct): opts_read{k} is aux.opts_read for reading the kth record, with defaults filled in
%     - opts_rays (cell array of struct): opts_rays{k} are the options used for creating the `ray structure` for the kth record
%     - opts_qpred (cell array of struct): opts_qpred{k} are the options used for creating the `quadratic form model` for the kth record along with
%
%         - var_tot (float): total variance of the coordinates
%         - var_ex (float 1-D array): variance explained by each dimension of the model
%         - predcoords (float 2-D array): predcoords(istim,idim) is the model coordinate for stimulus istim in dimension idim
%
%     - opts_check (struct): aux.opts_check, with defaults filled in
%     - syms_list (struct): structure with results of symmetry augmentation
%
%         - sym_apply (char): the symmetry applied, see 'sym_apply' above
%         - primary (int 1-D array): primary(k) is pointer to the data that was augmented by symmetry to yield the kth record
%         - aug (int 1-D array): aug(k) indicates the symmetry that was used to yield the kth record, e.g., if there are n_sym total symmetries (e.g., n_sym=8 for 'full'), then aug is in [1:n_sym]. The pairs [primary(k) aug(k)] are unique; 
%         a contiguous block of n_sym pairs share one value of primary(k) and have sequential values of aug(k)
%         - syms_applied (cell array of char): syms_applied{k} are the `binary texture` coordinates after symmetry transformation for record k
%
% Notes: Note regarding domain names, stimulus coordinates, and setup files
%     - The domain name is determined from the data file name based on the string preceding '_coords' (or, opts_read.coord_string, if customized).  This in turn determines how stimulus coordinates are specified.
%
%         - For non-reserved domain names (opts_read.type_class_aux, or any domain in opts_read.domain_list_def), stimulus coordinates are specified by 'type_coords' and no setup file is used.
%         - Other domain names are reserved, and stimulus coordinates are specified by a 'setup file', whose suggested name is constructed from the data file name up to the string preceding opts_read.coord_string, and appending opts_read.setup_suffix
%         - The domain name is also used to determine a 'type_class', which is used internally. This is carried out in psg_coorddata_parsename, as follows:
%
%             - The domains 'faces*' are reserved; type_class is faces_mpi; used with a setup file
%             - The domain 'irgb' is reserved; type_class is 'irgb'; used with a setup file
%             - The domain 'mater' is reserved; type_class is 'mater'; used with a setup file
%             - The domain opts_read.type_class_aux is NOT reserved; type class is type_class_aux; no setup file used
%             - Any domain that is listed in opts_read.domain_list_def is NOT reserved; type class is 'domain'; no setup file used
%
%         - Otherwise, type_class is set to opts_read.type_class_def, ('btc' unless customized); used with a setup file
%
%     - When a `dataset structure` is written by `rs_write_coordsets`, the setup file information, if any, will be embedded. When that file is read in and an embedded setup file is detected, no setup file is requested.
%     - Also see note below regarding customization
%
% Notes: Note regarding quadratic form model
%     - Quadratic form models may be used to generate a `coordinate structure` based on stimulus coordinates.
%     - The file must a cell array r, and each r{imodel} has at least has structure r The quadratic form model file must contain a field r, which must contain
%
%         - results.qfit (float 2-D array): a symmetric matrix indicating the quadratic form  which, when applied to stimulus coordinates, yields the model coordinates
%         - setup.label (char): a label for the model
%
%     - See demos/opposites_qform_example.mat for an example file with four quadratic form models, applicable to a stimulus set in which stimuli have three coordinates
%
% Notes: Note regarding customization
%     - The defaults for the following parameters can be set by editing the line containing generic.opts_read.[param_name] in  `rs_aux_defaults_define`, running it once, and saving the workspace as rs_aux_defaults.mat.
%
%         - if_gui
%         - ui_filter
%         - if_log
%         - if_warn
%         - if_auto
%         - data_fullname_def
%         - setup_fullname_def
%         - domain_list_def
%         - setup_suffix
%         - permutes_ok
%         - coord_string
%         - type_class_aux
%         - if_symaug
%         - sym_apply
%         - if_symaug_log
%
%     - The defaults for the following parameters can be set by editing the line containing generic.opts_qpred.[param_name] in  `rs_aux_defaults_define`, running it once, and saving the workspace as rs_aux_defaults.mat.
%
%         - qform_datafile_def
%         - qform_modeltype
%
%  See also: RS_AUX_CUSTOMIZE, RS_FINDRAYS, RS_CHECK_COORDSETS, PSG_GET_COORDSETS, PSG_COORDDATA_PARSENAME.
%
if (nargin<=1)
    aux=struct;
end
aux=filldefault(aux,'opts_read',struct);
type_coords_def=getfield(getfield(rs_aux_customize(setfield(struct(),'opts_import',struct()),'rs_import_coordsets'),'opts_import'),'type_coords_def');
aux.opts_read=filldefault(aux.opts_read,'type_coords_def',type_coords_def);
%
aux=filldefault(aux,'opts_rays',struct);
%
aux=filldefault(aux,'opts_qpred',struct);
%
aux=filldefault(aux,'nsets',0);
%
aux=filldefault(aux,'opts_check',struct);
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
aux=rs_aux_customize(aux,'rs_get_coordsets');
%
data_out=struct;
aux_out=struct;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
if ~iscell(fullnames)
    fullnames_list{1}=fullnames;
else
    fullnames_list=fullnames;
end
nsets=abs(aux.nsets);
nsets_named=length(fullnames_list);
if isempty(fullnames) | isempty(fullnames_list)
    nsets_named=0;
end
if nsets_named==0
    %If fullnames is empty, then set nsets to psg_get_coordsets as positive or negative
    if nsets>0
        if aux.opts_read.if_gui>0
            aux.nsets=-nsets; %flag for psg_get_coordsets to use gui
        else
            aux.nsets=nsets;
        end
    end
else %If fullnames is not empty, check that its length agrees with nsets and that each contains _coords
    if isempty(nsets)
        nsets=nsets_named;
        aux.nsets=nsets;
    end       
    if nsets_named~=nsets
        wmsg=sprintf('number of files listed (%3.0f) disagrees with number of files specified (%3.0f)',nsets_named,nsets);
        aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_read.if_warn));
    end
    for iset=1:nsets_named
        if ~contains(fullnames_list{iset},aux.opts_read.coord_string)
            wmsg=sprintf('file name %2.0f (%s) does not contain the required tag ''%s''',iset,fullnames_list{iset},aux.opts_read.coord_string);
            aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_read.if_warn));
        end
    end
    %attempt to read automatically
    aux.opts_read.data_fullnames=fullnames_list;
    %create setup files
    aux.opts_read.setup_fullnames=cell(1,nsets_named);
    for iset=1:nsets_named
        parsed=psg_coorddata_parsename(fullnames_list{iset},aux.opts_read);
        aux.opts_read.setup_fullnames{iset}=parsed.setup_fullname_def;
    end
end
if aux_out.warn_bad==0
    [sets,ds,sas,rayss,opts_read_used,opts_rays_used,opts_qpred_used,syms_list]=...
        psg_get_coordsets(aux.opts_read,aux.opts_rays,aux.opts_qpred,aux.nsets);
    %find rays via rs_findrays, so that coords are sought in type_coords or btc_augcoords
    for iset=1:nsets
        if strcmp(opts_read_used{iset}.type_class,'domain')
            [rayss{iset},wmsg,opts_rays_used{iset}]=rs_findrays(sas{iset},opts_read_used{iset}.setup_fullname,aux.opts_rays);
            if ~isempty(wmsg)
                aux_out=rs_warning(wmsg,0,aux_out);
            end
        end
    end
    %
    data_out.sets=sets;
    data_out.ds=ds;
    data_out.sas=sas;
    %
    aux_out.opts_check=aux.opts_check;
    aux_out.opts_read=opts_read_used;
    aux_out.opts_rays=opts_rays_used;
    aux_out.opts_qpred=opts_qpred_used;
    aux_out.rayss=rayss;
    aux_out.syms_list=syms_list;
    %
    %check internal consistency
    %
    for iset=1:nsets
        data_check=struct;
        data_check.ds{1}=data_out.ds{iset};
        data_check.sas{1}=data_out.sas{iset};
        data_check.sets{1}=data_out.sets{iset};
        check=rs_check_coordsets(data_check,setfield(aux.opts_check,'set_num_offset',iset-1));
        if ~isempty(check.warnings) %since strvcat([],[])~=[]
            aux_out.warnings=strvcat(aux_out.warnings,check.warnings);
            warn_leadin=getfield(getfield(rs_aux_customize(struct()),'overall'),'warn_leadin');
            for k=1:size(check.warnings,1)
                disp(cat(2,warn_leadin,check.warnings(k,:)));
            end
        end
        aux_out.warn_bad=aux_out.warn_bad+check.warn_bad;
    end
    %
else
    disp('cannot proceed');
    warn_leadin=getfield(getfield(rs_aux_customize(struct()),'overall'),'warn_leadin');
    for k=1:size(aux_out.warnings,1)
        disp(cat(2,warn_leadin,aux_out.warnings(k,:)));
    end
end
return
end

`rs_graphic_hints()`

graphic_hints=rs_graphic_hints() provides version-specific hints for graphics capabilities, including how alpha blending should be implemented

Output arguments:

graphic_hints ( struct ) –
descriptor of version-specific graphics capabilities, with fields
- if_alpha_color_line (int) – 1 if line color property allows for alpha as fourth component, otherwise 0
- if_alpha_color_line_marker (int) – 1 if if marker edge color property on a line allows for alpha as a fourth component, otherwise 0
- if_alpha_scatter_marker_edge (int) – 1 if MarkerEdgeAlpha is a property of Scatter, otherwise 0
- if_alpha_scatter_marker_face (int) – 1 if MarkerFaceAlpha is a property of Scatter, otherwise 0

`rs_import_coordsets(coords, aux)`

[data_out,aux_out]=rs_import_coordsets(data_in,aux) imports coordinates into a dataset structure with one record

This is the preferred method for bringing coordinates (as arrays) into the rs package, to create a dataset structure with one record, suitable for display and geometrical analysis.

If more than one record is to be combined into a single dataset structure, then rs_concat_coordsets can be used to combine several one-record dataset structures created here into a single dataset structure.
If the record is in a file, rs_read_coorddata can be used for a single record, or rs_get_coordsets for multiple files.
For multiple coordinate sets collected with the same experimental paradigm, use rs_get_coordsets, or use this routine and combine multiple dataset structures into a single dataset structure with rs_concat_coordsets.
No ray structure will be created; to do so, use rs_findrays
To createa a coordinate structure based on a quadratic form model, use rs_findrays and supply 'type_coords' to indicate the stimulus coordinates

Input arguments:

coords (float 2-D array, or cell array of float 2-D arrays) –

the coordinates; see note below regarding coordinates.
aux (struct) –
a structure, can be omitted, with fields
- opts_import (struct) – metadata, can be omitted, with fields listed below. Fields nstims, typenames, and type_coords are used to create the stimulus metadata structure; fields type, paradigm_type, paradigm_name,extra, subj_id, subj_id_short, label, and label_long are used to create the set metadata structure. Any or all can be omitted.
  - nstims (int) – number of stimuli; default determined from first non-empty entry in coords
  - typenames (cell array) – unique labels for stimuli; length should be equal to nstims; default is opts_import.typename_prefix followed by a sequential number, formatted with opts_import.typename_ndigits
  - typename_prefix (char) – prefix auto-generated stimulus names; default is 'type_'; see note below regarding customization
  - typename_ndigits (int) – number of digits in suffix for auto-generated stimulus names; default is 2; see note below regarding customization
  - type_coords (float 2-d array) – array with nstims rows specifing the stimulus coordinates; default is determined by opts_import.type_coords_def
  - type_coords_def (char) – method for auto-generation of stimulus coordinates; default is 'none'; see note below regarding customization
    - 'none': type_coords=[]
    - 'zeros': type_coords=zeros(nstims,1)
    - 'ones': type_coords=ones(nstims,1)
    - 'eye': type_coords=eye(nstims)
  - type (char) – overall category of coordinates; default is 'data'; alternatively, 'model' for coordinates originating in a computational model
  - paradigm_type (char) – overall category of experiment; default is 'unknown'; see note below regarding customization
  - paradigm_name (char) – subcategory of paradigm_type; default is opts_import.paradigm_type; see note below regarding customization
  - subj_id (char) – full subject identifier; default is 'unknown'; see note below regarding customization
  - subj_id_short (char) – short form of subj_id, e.g., for plot labels; default is opts_import.subj_id
  - extra (char) – free-form identifier; default is []; see note below regarding customization
  - label_long (char) – data source, typically a file path and name; default is 'unknown'; see note below regarding customization
  - label (char) – short form of label_long; default is opts_import.label_long; see note below regarding customization
- opts_check: options for consistency checking, with field
  - if_warn (int) – 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1

Output arguments:

data_out ( struct ) –
dataset structure with one record, and fields
- ds (singleton cell array) – coordinate structure, ds{1}{idim} is an array of [nstims idim] of coordinates
- sas (singleton cell array) – stimulus metadata structure, sas{1} is the stimulus metadata for the record
- sets (singleton cell array) – set metadata structure, sets{1} is the response metadata for the record
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_import (struct) – aux.opts_import, with defaults filled in
- opts_check (struct) – aux.opts_check, with defaults filled in

Note regarding coordinates

if 'coords' is a 2-dimensional array of size [nstims dmax], then coords(:,1:idim) is taken to be the coordinate set data_out.ds{1}{:,idim}, for each idim=1,...,dmax
if 'coords' is a cell array, then coords{idim} should be of size [nstims idim] or empty, and, if non-empty, is taken to be the coordinate set for dimension idim.

Note regarding customization

The default values of these parameters can be changed by editing rs_aux_defaults_define, running it once, and saving the workspace as rs_aux_defaults.mat. For example, to change the default paradigm type to 'cars', add or edit the line generic.opts_import.paradigm_type='cars' in the section in which generic.opts_import fields are defined.

typename_prefix
typename_ndigits
type_coords_def
paradigm_type
paradigm_name
subj_id
subj_id_short
extra
label_long
label

See also: rs_concat_coordsets, rs_aux_customize, rs_check_coordsets, PSG_TYPE_COORDS_DEF.

Source code in src/rs_import_coordsets.m

function [data_out,aux_out]=rs_import_coordsets(coords,aux)
% [data_out,aux_out]=rs_import_coordsets(data_in,aux) imports coordinates into a `dataset structure`  with one record
%
% This is the preferred method for bringing coordinates (as arrays) into the rs package, to create a `dataset structure` with one record, suitable for display and geometrical analysis.
%
%  - If more than one record is to be combined into a single `dataset structure`, then  `rs_concat_coordsets` can be used to
%    combine several one-record `dataset structures` created here into a single `dataset structure`.
%  - If the record is in a file, `rs_read_coorddata` can be used for a single record, or `rs_get_coordsets` for multiple files.
%  - For multiple coordinate sets collected with the same experimental paradigm, use `rs_get_coordsets`, or use this routine and combine multiple `dataset structures` 
%    into a single `dataset structure` with `rs_concat_coordsets`.
%  - No `ray structure` will be created; to do so, use rs_findrays
%  - To createa a `coordinate structure` based on a `quadratic form model`, use `rs_findrays` and supply 'type_coords' to indicate the stimulus coordinates
%
% Args:
%   coords (float 2-D array, or cell array of float 2-D arrays): the coordinates; see note below regarding coordinates.
%
%   aux (struct): a structure, can be omitted, with fields 
%
%     - opts_import (struct): metadata, can be omitted, with fields listed below.  Fields nstims, typenames, and type_coords are used
%     to create the `stimulus metadata structure`; fields type, paradigm_type, paradigm_name,extra, subj_id, subj_id_short, label, and label_long are used to create the `set metadata structure`.  Any or all can be omitted.
%
%         - nstims (int): number of stimuli; default determined from first non-empty entry in coords
%         - typenames (cell array): unique labels for stimuli; length should be equal to nstims; default is opts_import.typename_prefix followed by a sequential number, formatted with opts_import.typename_ndigits
%         - typename_prefix (char): prefix auto-generated stimulus names; default is 'type_'; see note below regarding customization
%         - typename_ndigits (int): number of digits in suffix for auto-generated stimulus names; default is 2; see note below regarding customization
%         - type_coords (float 2-d array): array with nstims rows specifing the `stimulus coordinates`; default is determined by opts_import.type_coords_def
%         - type_coords_def (char): method for auto-generation of `stimulus coordinates`; default is 'none'; see note below regarding customization
%
%             - 'none': type_coords=[]
%             - 'zeros': type_coords=zeros(nstims,1)
%             - 'ones': type_coords=ones(nstims,1)
%             - 'eye': type_coords=eye(nstims)
%
%         - type (char): overall category of coordinates; default is 'data'; alternatively, 'model' for coordinates originating in a computational model
%         - paradigm_type (char): overall category of experiment; default is 'unknown'; see note below regarding customization
%         - paradigm_name (char): subcategory of paradigm_type; default is opts_import.paradigm_type; see note below regarding customization
%         - subj_id (char): full subject identifier; default is 'unknown'; see note below regarding customization
%         - subj_id_short (char): short form of subj_id, e.g., for plot labels; default is opts_import.subj_id
%         - extra (char): free-form identifier; default is []; see note below regarding customization
%         - label_long (char): data source, typically a file path and name; default is 'unknown'; see note below regarding customization
%         - label (char): short form of label_long; default is opts_import.label_long; see note below regarding customization
%
%     - opts_check: options for consistency checking, with field
%
%         - if_warn (int): 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
% 
% Returns:
%   data_out (struct): `dataset structure` with one record, and fields
%
%     - ds (singleton cell array): `coordinate structure`, ds{1}{idim} is an array of [nstims idim] of coordinates
%     - sas (singleton cell array): `stimulus metadata structure`, sas{1} is the stimulus metadata for the record
%     - sets (singleton cell array): `set metadata structure`, sets{1} is the response metadata for the record
%
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_import (struct): aux.opts_import, with defaults filled in
%     - opts_check (struct): aux.opts_check, with defaults filled in
%
% Note: Note regarding coordinates
%    - if 'coords' is a 2-dimensional array of size [nstims dmax], then coords(:,1:idim) is taken to be the coordinate set data_out.ds{1}{:,idim},
%    for each idim=1,...,dmax
%    - if 'coords' is a cell array, then coords{idim} should be of size [nstims idim] or empty, and, if non-empty, is taken to be the coordinate set for dimension idim.
%
% Note: Note regarding customization
%    The default values of these parameters can be changed by editing `rs_aux_defaults_define`, running it 
%    once, and saving the workspace as rs_aux_defaults.mat.
%    For example, to change the default paradigm
%    type to 'cars', add or edit the line generic.opts_import.paradigm_type='cars'
%    in the section in which generic.opts_import fields are defined.
%
%    - typename_prefix
%    - typename_ndigits
%    - type_coords_def
%    - paradigm_type
%    - paradigm_name
%    - subj_id
%    - subj_id_short
%    - extra
%    - label_long
%    - label
%
%  See also: RS_CONCAT_COORDSETS, RS_AUX_CUSTOMIZE, RS_CHECK_COORDSETS, PSG_TYPE_COORDS_DEF.
%
if (nargin<=1)
    aux=struct;
end
%set up sub-structure options
aux=filldefault(aux,'opts_import',struct); %options for this module (psg_template)
%
aux=filldefault(aux,'opts_check',struct); %options for this module (psg_template)
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
aux=rs_aux_customize(aux,'rs_import');
%
aux.opts_import=filldefault(aux.opts_import,'nstims',0);
%
aux.opts_import=filldefault(aux.opts_import,'typenames',cell(0));
aux.opts_import=filldefault(aux.opts_import,'type_coords',[]);
%
aux.opts_import=filldefault(aux.opts_import,'type','data');
aux.opts_import=filldefault(aux.opts_import,'paradigm_name',aux.opts_import.paradigm_type);
aux.opts_import=filldefault(aux.opts_import,'subj_id_short',aux.opts_import.subj_id);
aux.opts_import=filldefault(aux.opts_import,'extra',[]);
aux.opts_import=filldefault(aux.opts_import,'label',aux.opts_import.label_long);
%
data_out=struct;
aux_out=struct;
aux_out.opts_import=aux.opts_import;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
set_fields_xfr={'type','paradigm_type','paradigm_name','subj_id','subj_id_short','extra','label_long','label'};
%
dim_list=[];
if iscell(coords)
    coords_use=coords;
else
    nds=size(coords,2);
    coords_use=cell(1,nds);
    for id=1:nds
        coords_use{id}=coords(:,[1:id]);
    end
end
ds=coords_use;
for k=1:length(coords_use)
    if ~isempty(coords_use{k})
        if aux.opts_import.nstims==0
            aux.opts_import.nstims=size(coords_use{k},1);
        end
        dim_list=[dim_list,size(coords_use{k},2)];
    end
end
nstims=aux.opts_import.nstims;
%
sas=struct;
sas.nstims=nstims;
if isempty(aux.opts_import.type_coords)
    sas.type_coords=psg_type_coords_def(nstims,aux.opts_import);
else
    sas.type_coords=aux.opts_import.type_coords;
end
sas.typenames=cell(nstims,1);
for istim=1:nstims
    if istim>length(aux.opts_import.typenames)
        sas.typenames{istim}=cat(2,aux.opts_import.typename_prefix,zpad(istim,aux.opts_import.typename_ndigits));
    else
        sas.typenames{istim}=aux.opts_import.typenames{istim};
    end
end
%
sets=struct;
sets.nstims=nstims;
sets.dim_list=dim_list;
for isf=1:length(set_fields_xfr)
    sets.(set_fields_xfr{isf})=aux.opts_import.(set_fields_xfr{isf});
end
sets.pipeline=[];
%
data_out=struct;
data_out.ds{1}=ds;
data_out.sas{1}=sas;
data_out.sets{1}=sets;
%
[check,opts_used]=rs_check_coordsets(data_out,aux.opts_check);
aux_out.warnings=check.warnings;
aux_out.warn_bad=check.warn_bad;
%
aux_out.opts_import=aux.opts_import;
aux_out.opts_check=aux.opts_check;
%
if aux_out.warn_bad>0
    disp('cannot proceed');
    data_out=struct();
end
return
end

`rs_knit_coordsets(data_in, aux)`

[data_out,aux_out]=rs_knit_coordsets(data_in,aux) knits a dataset structure into a consensus, and provides and displays statistics

Each of the records in the dataset structure data_in should contain the same stimuli, and in the same order, as determined by the strings in data_in.sas{k}.typenames for the record k. Missing data (e.g., for the stimulus s labeled by data_in.sas{k}.typenames{s}) should be indicated by NaN's in the row data_in.sets{k}{idim}(s,:), a row of length idim. The dataset structure returned by rs_align_coordsets meets these requirements.

Input arguments:

data_in (struct) –
dataset structure to be processed, with fields
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record
aux (struct) –
auxiliary inputs, may be omitted, with fields
- opts_knit (struct) – options for knitting and consistency checking, with fields to control
- opts_check (struct) – options for consistency checking, with field
  - if_warn (int) – 1 to show warnings when datasets are checked for consistency or sufficient overlaps, 0 to suppress; default is 1
- opts_pca (struct) – options for principal components analysis of consensus, typically omitted, only relevant if if_pca=1
- opts_align (struct) – options for alignment of data, typically omitted; see note below regarding recalculation of alignment
- opts_rays (struct) – options for rays, typically omitted; see note below regarding rays
- sa_pooled (struct) – include to avoid recalculation of alignment, otherwise omit; see note below regarding recalculation of alignment
- data_align (struct) – include to avoid recalculation of alignment, otherwise omit; see note below regarding recalculation of alignment

Output arguments:

data_out ( struct ) –

dataset structure with a single record consisting of the consensus coordinates from data_in, same format as as data_in
aux_out ( struct ) –
auxiliary outputs and parameter values used
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_knit (struct) – aux.opts_knit, with defaults filled in
- opts_check (struct) – aux.opts_check, with defaults filled in
- opts_pcon (cell array of struct) – opts_pcon{idim} are the options used in Procrustes alignment for model dimension idim
- opts_pca (struct) – aux.opts_pca, with defaults filled in
- opts_align (struct) – aux.opts_align, with defaults filled in
- opts_rays (cell array with one element) – opts_rays{1} is a structure which contains the options used for creating rays in data_out
- rayss (cell array with one element) – rayss{1} is the ray structure for data_out; see note below regarding rays
- coords_havedata (int 2-D array) – coords_havedata(s,k)=1 if the stimulus data_out.sets{:}.typenames{s} is present and not NaN in record k of data_in, 0 otherwise
- components (struct) – a dataset structure, the kth record corresponds to the kth record of data_in after transformation to the consensus; all stimuli in data_out will be included but coordinates for stimuli not in data_in.sas{k} will be NaN
- knit_stats (struct) – statistics of knitting; see notes below regarding statistics and replotting a previous analysis
- knit_stats_setup (struct) – parameters extracted from aux.opts_knit, along with the additional fields below; see note below regarding replotting a previous analysis
  - nsets (int) – number of records in data_in
  - nstims (int) – number of stimuli
  - dataset_labels (cell array of char) – dataset labels, from data_in.sets{:}.label
  - stimulus_labels (cell array of char) – stimulus labels, from data_out.sas{1}.typenames
- fig_handle (handle) – handle to figure created (present only if statistics are plotted)
- ts_pca (cell array) – ts_pca{idim}{k} is the transformation from data_in.ds{k}{idim} to the consensus, taking into account final pca; present only if aux.opts_knit.if_pca=1 The transformation is ts_pca{idim,k}.scaling*[data_in{k}{idim}]*ts_pca{idim}{k}.orthog+ts_pca{idim}{k}.translation. Note that translation may be nonzero even if allow_offset=0, since this is the best transformation from the original data to the consensus -- which may include a translation even if the consensus is calculated wthout allowing translation. If no translation is desired, then set this field to zeros(1,idim). If dim_list_out>dim_list_in, then data_in{k}{idim} needs to be right-padded by columns of zeros for missing dimensions.
- details (cell array of struct) – details{idim} contains details of the convergence towards knitting for data_in.ds{:}{idim}; present only if aux.opts_knit.keep_details=1; fields include
  - ts_cum (cell array) – ts_cum{m}{k} is the transformation found from record k, i.e., from data_in.ds{k}(istim,:), to the current consensus, on iteration m
  - rms_change (float 1-D array) – rms_change(m) is the rms change of the consensus coordinates from the previous iteration
  - consensus (float 3-D array) – consensus(istim,:,m) are theconsensus coordinates on iteration m
  - z (float 4-D array) – consensus(istim,:,k,m) are the coordinates data_in.ds{k}(istim,:) transformed to match the consensus on dimension m
  - rms_dev (float 2-D array) – rms_dev(k,m) is the rms deviation of record k from the consensus, after the current transformation
  - initial_guess (float 2-D array) – initial_guess(istim,idim) are the coordinates of stimulus istim used for the initial guess for the consensus
  - alignment (float 2-D array) – alignment(istim,idim) are the coordinates of stimulus istim used to align the consensus
  - initialize_use (int 1-D array) – initialize_use(istim,1) is the reocrd is used to initialize each point (if initialize_set>0)
  - swarm (struct) – details from swarm optimization, present only if opts_knit.if_swarm=1 (see note regarding swarm optimization), with fields
    - rms_dev_total_min (float) – root-mean-squared deviation for best alignment from preliminary swarm runs
    - rms_dev_total_definitive (float) – root-mean-squared deviation for best alignment for definitive run
    - class_name_min (char) – the class that produced the best alignment in preliminary runs
    - itry_min (int) – the instance in the class that produced the best alignment in preliminary runs
    - opts_definitive (struct) – the options in opts_knit that produced the best alignment in preliminary runs
    - if_def_eq_swarm (int) – 1 if halting parameters (max_niters, max_rmstol) for preliminary and definitive runs match
    - init_components (struct) – results of swarm optimizations based on component datasets
    - init_pca (struct) – results of swarm optimizations based on principal components
    - init_random (struct) – results of swarm optimizations based on random starts

Note regarding propagation of fields from data_in to data_out

For all records with data_in.sets{k}.type='data', the strings in data_in.sets{k}.paradigm_type must agree.
The 'type' field of data_in.sets{1} is propagated to data_out.sets{1}
If present, the 'paradigm_type' field common to data_in.sets{:} for all k for which data_in.sets{k}.type='data' is propagated to data_out.sets{1}
The following fields of data_in.sets{:} are concatenated with + signs in between and propagated to data_out.sets{1}
- paradigm_name
- subj_id
- subj_id_short
- label_long
- label
- extra
pipeline field of data_out.sets{1}:
- data_out.sets{1}.pipeline.type='knit'.
- data_out.sets{1}.pipeline.sets_combined{:} contains metadata from all records of data_in

Note regarding statistics and plots

If aux.opts_knit.if_stats=1, variance explained by the consensus coordinates are calculated and returned in aux_out.knit_stats, in the following fields (rmsd=root-mean-squared deviation):
- rmsdev_overall (float 1-D array): rmsdev_overall(idim) is the rmsd across all records and stimuli
- rmsdev_setwise (float 2-D array): rmsdev_setwise(idim,k): rmsd within record k, across stimuli
- rmsdev_stmwise (float 2-D array): rmsdev_stmwise(idim,istim): rmsd within stimulus istim, across records
The counts for each of these calculations are counts_[overall|setwise|stmwise], and the available rmsd (from zero) is given by rmsavail_[overall|setwise|stimwise].
If aux.opts_knit.nshuffs>0 (default is 500), then a parallel computation is done after random shuffles of the stimulus labels within each record, and the results are returned in rmsdev_[overall|setwise|stimwise]_shuff.
For the shuffled quantities, the first two dimensions are the same as the unshuffled quantities; dimension 3 is always 1; dimension 4 (length: nshuffs) is which shuffle; dimension 5 (length: 2) is the mode: 1 for last coordinate only shuffled, 2 for all coordinates shuffled. To control whether the same random number seed is used on each run, use aux.opts_knit.if_frozen (default is 1).
if aux.opts_knit.if_plot=1 (default if if_stats=1), then a figure is created, with four panels:
- a heatmap of rmsdev_setwise as a function of dimension
- a heatmap of rmsdev_stmwise as a function of dimension
- a comparison of the unexplained portion of rmsdev_overall (black) to quantiles of rmsdev_overall_shuff (mode 1: magenta, mode 2: red). Quantiles are specified by shuff_quantiles; if nshuffs=0, shuffled values will not be plotted
- a comparison of the explained rmsd, parallel to the above, with available rmsd in blue
Other fields in aux_out.knit_stats are the following. Note that ds_components and ts are analogus to the fields components and ts_pca] in aux_out, except that ds_components and ts do not include the rotation to principal components (if requested by aux.opt_knit.if_pca=1). This is not needed for the variance statistics. For a transformation that includes the rotation, see aux_out.ts_pca.
- opts_pcon (struct): supplied options for Procrustes consensus algorithm
- opts_pcon_eachdim (cell array of struct): opts_pcon_eachdim{idim} are the options used for dimension idim
- ds_knitted (cell array): ds_knitted{idim} are the consensus coordinates
- ds_components (cell array): ds_components{k}{idim} are the coordinates for record k that most closely match the consensus
- ts (cell array): ts{idim}{k} is the Procrustes transformation for record k. The transformation is [ds_components{k}{idim}]=ts{idim}{k}.scaling*[data_in.ds{k}{idim}]*ts{idim}{k}.orthog+ts{idim}{k}.translation; see transformation structures for further details. If dim_list_out>dim_list_in, then data_in.ds{k}{idim}] needs to be right-padded by columns of zeros for missing dimensions.

Note regarding recalculation of alignment

The first step in forming a consensus is alignment, which identifies the common stimuli among the records of data_in, and to place them in the same order. By default, this is carried out in rs_knit_coordsets by a call to rs_align_coordsets, using options aux.align_opts. This recalculation can be avoided by supplying aux_out from a previous call to rs_align_coordsets, as follows: aux.sa_pooled=aux_out.sa_pooled, aux.data_align=aux_out.data_out

Note regarding Procrustes consensus algorithm

To find a consensus set of coordinates, the coordinates in each record of data_in are rotated, and optionally translated (if allow_offset=1), scaled (if allow_scale=1), and reflected (if allow_reflection=1). These transformations are carried out for separately for each set dimension for which coordinates are present in all of the records, i.e., for which data_in.ds{k}{idim} exists for all k.
The algorithm, in procrustes_consensus.m, is iterative. Briefly, after an initial guess is determined, a Procrustes transformation is found that minimizes the rms deviation between each record and the current guess. The guess is then revised by setting each stimulus' coordinates equal to the centroid of the coordinates of that stimulus across the records. To avoid drift of the updated guess, it is Procrustes-transformed for closest match to an alignment coordinate set (the alignment set, unless otherwise specified, is equal to the initial guess).
The iteration ends when either the number of iterations exceeds max_niters (default=1000), or the rms change of the guess is less than max_rmstol (default=10^-5)
There are several choices for initialization and alignment.
- For most purposes, the default initialization method (pcon_init_method=0) can be used, which uses the principal components of all the stimulus coordinates in all of the records. These can be optionally forced to be centered (pcon_init_method=-1) or not (pcon_init_method=-2); if unspecified (default), centering is determined by allow_offset. For these choices, if_initpca_rot=1 rotates the initial guess to match the data, or not. The default for if_initpca_rot is 1 unless any of dim_list_out>dim_list_in, in which case it is 0. The heuristic for not rotating if dim_list_out>dim_list_in, i.e., two or more sets of coordinates are to be knit together to construct a coordinate set with a greater number of dimensions, is that without rotation, the principal components reflect projections of the coordinates that are present in any of the records.
- Alternatively, pcon_init_method=r, r>0, specifies that the coordinates in data_in{r}{idim} are used.
- If pcon_init_method='specify', then pcon_initial_guess{idim} is an array of size [npts ids] for the initial guess, and pcon_alignment{idim}, which defaults to pcon_initial_guess, is used for the alignment at the end of each iteration. pcon_initial_guess and pcon_alignment may be omitted, in which case random values are used. To control whether the same random number seed is used on each run, use if_frozen (default is 1, which uses the same random numbers on each run).
- The solution is only unique up to rotation (and translation and reflection, if these components are allowed). The ambiguity is resolved by matching the consensus solution to the initial guess (or, pcon_alignment{idim} if separately supplied with pcon_init_method=0), as described above.
- Under some circumstances (e.g., several solutions that are nearly equally good), the solution found by the algorithm may depend on the initialization choice. A simple strategy to check for this is to compare the results with the default pcon_init_method=0 to the results with pcon_init_method='specify' and if_frozen=0. There are two ways that this dependency can happen.
  - One is that the number of overlapping stimuli is too small. For example, at least m points are required to determine a rotation and translation in an m-dimensional space; if there are fewer overlaps, then a consensus will still be found but there are many other consensus datasets that fit equally well. Note that setting aux.opts_knit.if_dim_heuristics=1 will display heuristics concerning whether there are sufficient overlaps.
  - A second way is that there are a sufficient number of points, but there are several solutions that are approximately equally good. Under these circumstances, the algorithm may get stuck in a local minimum. This possibility only occurs when there are at least three records in data_in, as the procedure reduces to the standard Procrustes algorithm, which finds the consensus when there are only two records, is deterministic other than does rotational ambiguity.
The swarm option (opts_knit.if_swarm=1) can be used to automatically step through several types of initialization methods, and choose the best.
If allow_scale=1, then the optimal solution will depend not only on the data points that overlap, but also on the data points that are unique to individual components, since these contribute to determining tthe rescaling on each iteration

Note regarding replotting a previous analysis

To replot a previous calculation with additional customization or to make a composite figure, proceed as follows:
- data_in should be equal to that used in the previous calculation.
- aux.knit_stats should be equal to aux_out.knit_stats from the previous calculation
- aux.knit_stats_setup should be equal to aux_out.knit_stats_setup from the previous calculation
The following modifications are allowed in fields of knit_stats_setup:
- dataset_labels (cell array of char): dataset labels; default is data_in.sets{:}.label
- stimulus_labels (cell array of char): stimulus labbels; default is data_out.sas{1}.typenames
- shuff_quantiles (float 1-d array): quantiles to show; default is 0.01 0.05 0.5 0.95 0.99
- fig_handle (handle): figure handle to plot into; figure will be created if not supplied
- row (int): row to plot into; default is 1
- nrows (int): number of rows in the figure; default is row
No further calculations are done
On return, data_out will be empty, and aux_out.fig_handle will be the figure handle
In creating a composite figure, rows should be plotted in order from top to bottom, as plotting the bottom row triggers an equalization of the color scale. See rs_knit_coordsets_demo for an example.

Note regarding swarm optimization

This is enabled by setting opts_knit.if_swarm=1.
Preliminary runs are made with several classes of initial guesses, and stopping parameters based on max_niters and max_rmstol in opts_knit.swarm
The choice that leads to the smallest rms deviation of the aligned coordinates is recorded in aux.details{idim}.swarm, and used for a definitive run, using the stopping parameters in opts_knit
The classes are
- init_components: the original component datasets are used; variants choose each component dataset in succession as the initial guess, advancing to the next dataset if a stimulus is missing
- init_pca: principal components of the datasets are used, with options (mean subtraction, rotation) as described in the note regarding the Procrustes alignment algorithm; variants consist of computing the principal components from the coordinates of stimuli present in increasing numbers of the datasets
- init_random: Gaussian random varibles are used as the initial guess; the number of variants is given by opts_knit.swarm.nrand; default is 10; these random numbers do not affect the random numbers used for permutation testing
Results from each class are recorded in aux.details{idim}.(class_name), including
- counts: the number of variants for each class
- rms_dev_totals: the rms deviation for each example of the class
- range: the range of rms deviations for each class
- itry_min: the optimal choice within each class
- opts: the options usef for the variant with the lowest rms deviation

Note regarding rays

The ray structure describes relationships among the simulus coordinates: rays, i.e., sets of stimuli that lie along an axis or a ray from the origin, rings, stimuli that lie at an approximately equal distance from the origin, and nearest neighbors. It is only created if there is a valid set of stimulus coordinates.

See also

rs_align_coordsets, rs_aux_customize, rs_check_coordsets, rs_findrays, PSG_ALIGN_COORDSETS, PSG_KNIT_STATS, PSG_REMNAN_COORDSETS, PSG_COORD_PIPE_UTIL, overlap_heuristics, procrustes_consensus, PSG_ALIGN_STATS_PLOT.

Source code in src/rs_knit_coordsets.m

function [data_out,aux_out]=rs_knit_coordsets(data_in,aux)
% [data_out,aux_out]=rs_knit_coordsets(data_in,aux)
% knits a `dataset structure` into a consensus, and provides and displays statistics
%
% Each of the records in the `dataset structure` `data_in` should contain the same stimuli, 
% and in the same order, as determined by the strings in data_in.sas{k}.typenames for the record k.
% Missing data (e.g., for the stimulus s labeled by data_in.sas{k}.typenames{s}) should be indicated by 
% NaN's in the row data_in.sets{k}{idim}(s,:), a row of length idim.  The `dataset structure` returned by `rs_align_coordsets` 
% meets these requirements.
%
% Args:
%   data_in (struct): `dataset structure` to be processed, with fields
%
%     - ds (cell array): `coordinate structure`, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
%     - sas (cell array): `stimulus metadata structure`, sas{k} is the stimulus metadata for the kth record
%     - sets (cell array): `set metadata structure`, sets{k} is the response metadata for the kth record
%
%   aux (struct): auxiliary inputs, may be omitted, with fields
%
%     - opts_knit (struct): options for knitting and consistency checking, with fields to control
%
%         - **Transformations**
%         - allow_offset (int): 1 to allow translational offset, 0 does not allow; default is 1
%         - allow_scale (int): 1 to allow scaling of each dataset into the consensus, 0 does not allow; default is 0
%         - allow_reflection (int): 1 to allow reflection, 0 does not allow; default is 1
%         - if_normscale (int): 1 to normalize consensus to size of `data_in` (determined by geometric mean of scale factors for each dataset), 0 does not (no effect if allow_scale=0); default is allow_scale
%         - if_pca (int): 1 to rotate the consensus coordinates in data_out into its principal components, 0 does not; default is 0
%
%         - **Statistics and shuffles**
%         - if_stats (int): 1 to do statistics of variance explained, 0 does not; default is 0
%         - nshuffs (int): number of shuffles for calculating statistics; default is 500 if if_stats=1, 0 if if_stats=0; see note below regarding statistics and plots
%         - if_frozen (int): random number control for shuffles and initialization; 1 for same numbers every run, 0 for different random numbers each run, negative integer for a fixed seed each run; default is 1; see notes below regarding statistics and Procrustes consensus algorithm
%
%         - **Dimension selection**
%         - dim_max_in (int): maximum dimension of data_in.ds to use; default is maximum available across all datasets
%         - dim_list_in (int 1-D array): list of dimensions to use from data_in.ds, must be unique; default is [1:dim_max_in]
%         - dim_aug (int): number of dimensions to add for knitted datasets in data_out.ds; default is 0; see note below regarding Procrustes consensus algorithm
%         - dim_list_out (int 1-D array): list of dimensions  to create in data_out.ds, must be same length as dim_list_in, no less than corresponding elements of dim_list_in, and unique; default is [1:dim_list_in]+dim_aug
%         - if_dim_heuristics (int): 1 to always show heuristics as to whether there are sufficient constraints for the requested dimensions, 0 only if insufficient constraints; default is 0
%
%         - **Plotting and replotting**
%         - if_plot (int): 1 to plot statistics, 0 does not; default is if_stats
%         - shuff_quantiles (float 1-D array): quantiles to plot; default is [0.01 0.05 0.5 0.95 0.99]
%         - if_remove_path_label (int): 1 to remove path from filename when used as a label (in data_in.sets{:}.label), 0 does not; default is 1
%         - knit_stats (struct): include to replot a previous analysis, otherwise omit; see note below regarding replotting
%         - knit_stats_setup (struct): include to replot a previous analysis, otherwise omit; see note below regarding replotting
%
%         - **Logging and optimization**
%         - if_log (int): 1 to log progress, 0 to suppress; default is 1
%         - pcon_init_method (int or char): typically omitted; default is 0, leading to 'pca' method for initializing `procrustes_consensus`; see note below regarding Procrustes consensus algorithm for other options
%         - if_swarm (int): 1 to enable swarm optimization (overriding pcon_init_method), 0 to disable; default is 0; see note below regarding swarm optimization
%         - if_initpca_rot (int): typically omitted, default is 1 unless any of dim_list_out>dim_list_in; see note below regarding Procrustes consensus algorithm
%         - max_iters (int): maximum number of iterations for Procrustes consensus; default is 1000; see note below regarding Procrustes consensus algorithm
%         - max_rmstol (float): maximum change ofcoordinates for consensus solution; default is 10^-5; see note below regarding Procrustes consensus algorithm
%         - keep_details (int): 1 to return details of Procrustes consensus minimization, 0 does not; default is 0; see note below regarding Procrustes consensus algorithm
%         - pcon_initial_guess (cell array): specified initial guess for Proccrustes minimization, typically omitted; see note below regarding Procrustes consensus algorithm
%         - pcon_alignment (cell array): specified alignment for Procrustes minimization, typically omitted; see note below regarding Procrustes consensus algorithm
%         - swarm (struct): if present, sets options for swarm optimization (see note below regarding swarm optimization); fields are
%
%             - if_log (int): 1 for logging swarm optimization, 0 to suppress; default is 0
%             - if_frozen (int):   1 for same numbers each run, 0 for different random numbers each run, negative integer for a fixed seed each run; default is 1
%             - max_niters (int):  maximum number of iterations to determine best initialization for swarm; default is 50
%             - max_rmstol (float): tolerance for stopping iterations to dtermine best initializatio for swarm; default is 10^-4
%             - nrand (int): number of initial guesses based on random numbers; default is 10
%
%     - opts_check (struct): options for consistency checking, with field
%
%         - if_warn (int): 1 to show warnings when datasets are checked for consistency or sufficient overlaps, 0 to suppress; default is 1
%
%     - opts_pca (struct): options for principal components analysis of consensus, typically omitted, only relevant if if_pca=1
%     - opts_align (struct): options for alignment of data, typically omitted; see note below regarding recalculation of alignment
%     - opts_rays (struct): options for rays, typically omitted; see note below regarding rays
%
%     - sa_pooled (struct): include to avoid recalculation of alignment, otherwise omit; see note below regarding recalculation of alignment
%     - data_align (struct): include to avoid recalculation of alignment, otherwise omit; see note below regarding recalculation of alignment
% 
% Returns:
%   data_out (struct): `dataset structure` with a single record consisting of the consensus coordinates from `data_in`, same format as  as `data_in`
%
%   aux_out (struct): auxiliary outputs and parameter values used
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_knit (struct): aux.opts_knit, with defaults filled in
%     - opts_check (struct): aux.opts_check, with defaults filled in
%     - opts_pcon (cell array of struct): opts_pcon{idim} are the options used in Procrustes alignment for model dimension idim
%     - opts_pca (struct): aux.opts_pca, with defaults filled in
%     - opts_align (struct): aux.opts_align, with defaults filled in
%     - opts_rays (cell array with one element): opts_rays{1} is a structure which contains the options used for creating rays in data_out
%     - rayss (cell array with one element): rayss{1} is the `ray structure` for data_out; see note below regarding rays
%     - coords_havedata (int 2-D array): coords_havedata(s,k)=1 if the stimulus data_out.sets{:}.typenames{s} is present and not NaN in record k of `data_in`, 0 otherwise
%     - components (struct): a `dataset structure`, the kth record corresponds to the kth record of `data_in` after transformation to the consensus; all stimuli in
%     data_out will be included but coordinates for stimuli not in data_in.sas{k} will be NaN
%     - knit_stats (struct): statistics of knitting; see notes below regarding statistics and replotting a previous analysis
%     - knit_stats_setup (struct): parameters extracted from aux.opts_knit, along with the additional fields below; see note below regarding replotting a previous analysis
%
%         - nsets (int): number of records in `data_in`
%         - nstims (int): number of stimuli
%         - dataset_labels (cell array of char): dataset labels, from data_in.sets{:}.label
%         - stimulus_labels (cell array of char): stimulus labels, from data_out.sas{1}.typenames
%
%     - fig_handle (handle): handle to figure created (present only if statistics are plotted)
%     - ts_pca (cell array): ts_pca{idim}{k} is the transformation from data_in.ds{k}{idim} to the consensus, taking into account final pca; present only if aux.opts_knit.if_pca=1
%     The transformation is
%     ts_pca{idim,k}.scaling*[data_in{k}{idim}]\*ts_pca{idim}{k}.orthog+ts_pca{idim}{k}.translation. Note that translation may be nonzero even if allow_offset=0, since this 
%     is the best transformation from the original data to the consensus -- which may include a translation even if the consensus is calculated
%     wthout allowing translation. If no translation is desired, then set this field to zeros(1,idim).
%     If dim_list_out>dim_list_in, then data_in{k}{idim} needs to be right-padded by columns of zeros for missing dimensions.
%     - details (cell array of struct): details{idim} contains details of
%     the convergence towards knitting for data_in.ds{:}{idim}; present only if aux.opts_knit.keep_details=1; fields include
%
%         - ts_cum (cell array): ts_cum{m}{k} is the transformation found
%         from record k, i.e., from data_in.ds{k}(istim,:), to the current consensus, on iteration m
%         - rms_change (float 1-D array): rms_change(m) is the rms change of the consensus coordinates from the previous iteration
%         - consensus (float 3-D array): consensus(istim,:,m) are theconsensus coordinates on iteration m
%         - z (float 4-D array): consensus(istim,:,k,m) are the coordinates
%         data_in.ds{k}(istim,:) transformed to match the consensus on dimension m
%         - rms_dev (float 2-D array): rms_dev(k,m) is the rms deviation of record k from the consensus, after the current transformation
%         - initial_guess (float 2-D array): initial_guess(istim,idim) are the coordinates of stimulus istim used for the initial guess for the consensus
%         - alignment (float 2-D array): alignment(istim,idim) are the coordinates of stimulus istim used to align the consensus
%         - initialize_use (int 1-D array): initialize_use(istim,1) is the reocrd is used to initialize each point (if initialize_set>0)
%         - swarm (struct): details from swarm optimization, present only if opts_knit.if_swarm=1 (see note regarding swarm optimization), with fields
%
%             - rms_dev_total_min (float): root-mean-squared deviation for best alignment from preliminary swarm runs
%             - rms_dev_total_definitive (float): root-mean-squared deviation for best alignment for definitive run
%             - class_name_min (char): the class that produced the best alignment in preliminary runs
%             - itry_min (int): the instance in the class that produced the best alignment in preliminary runs
%             - opts_definitive (struct): the options in opts_knit that produced the best alignment in preliminary runs
%             - if_def_eq_swarm (int): 1 if halting parameters (max_niters, max_rmstol) for preliminary and definitive runs match
%             - init_components (struct): results of swarm optimizations based on component datasets
%             - init_pca (struct): results of swarm optimizations based on principal components
%             - init_random (struct): results of swarm optimizations based on random starts
%
% Note: Note regarding propagation of fields from data_in to data_out
%     - For all records with data_in.sets{k}.type='data', the strings in data_in.sets{k}.paradigm_type must agree.
%     - The 'type' field of data_in.sets{1} is propagated to data_out.sets{1}
%     - If present, the 'paradigm_type' field common to data_in.sets{:} for all k for which data_in.sets{k}.type='data' is propagated to data_out.sets{1}
%     - The following fields of data_in.sets{:} are concatenated with + signs in between and propagated to data_out.sets{1}
%
%         - paradigm_name
%         - subj_id
%         - subj_id_short
%         - label_long
%         - label
%         - extra
%
%     - pipeline field of data_out.sets{1}:
%
%         - data_out.sets{1}.pipeline.type='knit'.
%         - data_out.sets{1}.pipeline.sets_combined{:} contains metadata from all records of `data_in`
%
% Note: Note regarding statistics and plots
%     - If aux.opts_knit.if_stats=1, variance explained by the consensus coordinates are calculated and returned in aux_out.knit_stats, in the following fields (rmsd=root-mean-squared deviation):
%
%         - rmsdev_overall (float 1-D array): rmsdev_overall(idim) is the rmsd across all records and stimuli
%         - rmsdev_setwise (float 2-D array): rmsdev_setwise(idim,k): rmsd within record k, across stimuli
%         - rmsdev_stmwise (float 2-D array): rmsdev_stmwise(idim,istim): rmsd within stimulus istim, across records
%
%     - The counts for each of these calculations are counts_[overall|setwise|stmwise], and the available rmsd (from zero) is given by rmsavail_[overall|setwise|stimwise].
%     - If aux.opts_knit.nshuffs>0 (default is 500), then a parallel computation is done after random shuffles of the stimulus labels within each record,
%     and the results are returned in rmsdev_[overall|setwise|stimwise]_shuff.
%     - For the shuffled quantities, the first two dimensions are the same as the unshuffled quantities; dimension 3 is
%     always 1; dimension 4 (length: nshuffs) is which shuffle; dimension 5 (length: 2) is the mode: 1 for last coordinate only shuffled, 2 for all coordinates shuffled.
%     To control whether the same random number seed is used on each run, use aux.opts_knit.if_frozen (default is 1).
%     - if aux.opts_knit.if_plot=1 (default if if_stats=1), then a figure is created, with four panels:
%
%         - a heatmap of rmsdev_setwise as a function of dimension
%         - a heatmap of rmsdev_stmwise as a function of dimension
%         - a comparison of the unexplained portion of rmsdev_overall (black) to quantiles of
%         rmsdev_overall_shuff (mode 1: magenta, mode 2: red). Quantiles are specified by shuff_quantiles; if nshuffs=0, shuffled values will not be plotted
%         - a comparison of the explained rmsd, parallel to the above, with available rmsd in blue
%
%     - Other fields in aux_out.knit_stats are the following.  Note that ds_components and ts are analogus to the fields components and ts_pca] in aux_out, except that ds_components and ts do not include the rotation to principal components (if
%     requested by aux.opt_knit.if_pca=1). This is not needed for the variance statistics. For a transformation that includes the rotation, see aux_out.ts_pca.
%
%         - opts_pcon (struct): supplied options for Procrustes consensus algorithm
%         - opts_pcon_eachdim (cell array of struct): opts_pcon_eachdim{idim} are the options used for dimension idim
%         - ds_knitted (cell array): ds_knitted{idim} are the consensus coordinates
%         - ds_components (cell array): ds_components{k}{idim} are the coordinates for record k that most closely match the consensus
%         - ts (cell array): ts{idim}{k} is the Procrustes transformation for record k. The transformation is
%         [ds_components{k}{idim}]=ts{idim}{k}.scaling\*[data_in.ds{k}{idim}]*ts{idim}{k}.orthog+ts{idim}{k}.translation; see `transformation structures` for further details.
%         If dim_list_out>dim_list_in, then data_in.ds{k}{idim}] needs to be right-padded by columns of zeros for missing dimensions.
%     
% Note: Note regarding recalculation of alignment
%     The first step in forming a consensus is alignment, which identifies the common stimuli among the records of `data_in`, and to 
%     place them in the same order. By default, this is carried out in rs_knit_coordsets by a call to rs_align_coordsets, using options aux.align_opts.
%     This recalculation can be avoided by supplying aux_out from a
%     previous call to rs_align_coordsets, as follows: aux.sa_pooled=aux_out.sa_pooled, aux.data_align=aux_out.data_out
%
% Note: Note regarding Procrustes consensus algorithm
%     - To find a consensus set of coordinates, the coordinates in each record of `data_in` are rotated, and optionally translated (if allow_offset=1),
%     scaled (if allow_scale=1), and reflected (if allow_reflection=1). These transformations are carried out for separately for each set dimension
%     for which coordinates are present in all of the records, i.e., for which data_in.ds{k}{idim} exists for all k.
%     - The algorithm, in procrustes_consensus.m, is iterative.  Briefly, after an initial guess is determined, a Procrustes 
%     transformation is found that minimizes the rms deviation between each record and the current guess. The guess is then
%     revised by setting each stimulus' coordinates equal to the centroid of the coordinates of that stimulus across the records.  To avoid drift of the updated guess, it is Procrustes-transformed for closest match
%     to an alignment coordinate set (the alignment set, unless otherwise specified, is equal to the initial guess).
%     - The iteration ends when either the number of iterations exceeds max_niters (default=1000),
%     or the rms change of the guess is less than max_rmstol (default=10^-5)
%     - There are several choices for initialization and alignment.
%
%         - For most purposes, the default initialization method (pcon_init_method=0) can be used, which uses the principal components of all the stimulus coordinates in all of the records.
%         These can be optionally forced to be centered (pcon_init_method=-1) or not (pcon_init_method=-2); if unspecified (default), centering is determined by allow_offset.
%         For these choices, if_initpca_rot=1 rotates the initial guess to match the data, or
%         not. The default for if_initpca_rot is 1 unless any of dim_list_out>dim_list_in, in which case it is 0.
%         The heuristic for not rotating if dim_list_out>dim_list_in, i.e., two or more sets of coordinates are to be knit together to construct a coordinate set with a greater number of dimensions,
%         is that without rotation, the principal components reflect projections of the coordinates that are present in any of the records.
%         -  Alternatively, pcon_init_method=r, r>0, specifies that the coordinates in data_in{r}{idim} are used.
%         -  If pcon_init_method='specify', then pcon_initial_guess{idim} is an array of size [npts ids] for the
%         initial guess, and pcon_alignment{idim}, which defaults to
%         pcon_initial_guess, is used for the alignment at the end of each iteration.  pcon_initial_guess and pcon_alignment may be omitted, in which case random values are used.
%         To control whether the same random number seed is used on each run, use if_frozen (default is 1, which uses the same random numbers on each run).
%         - The solution is only unique up to rotation (and translation and reflection, if these components are allowed).  The ambiguity is resolved by
%         matching the consensus solution to the initial guess (or, pcon_alignment{idim} if separately supplied with pcon_init_method=0), as described above.
%         - Under some circumstances (e.g., several solutions that are nearly equally good), the solution found by the algorithm may depend on
%         the initialization choice.  A simple strategy to check for this is to compare the results with the default pcon_init_method=0 to the results with
%         pcon_init_method='specify' and if_frozen=0. There are two ways that this dependency can happen.
%
%             - One is that the number of overlapping stimuli is too small. For example,
%             at least m points are required to determine a rotation and translation in an m-dimensional space; if there are fewer overlaps, then a consensus will
%             still be found but there are many other consensus datasets that fit equally well. Note that setting aux.opts_knit.if_dim_heuristics=1 will display heuristics concerning whether there are sufficient overlaps.
%             - A second way is that there are a sufficient number of points, but there are several solutions that are approximately equally good. 
%             Under these circumstances, the algorithm may get stuck in a local minimum. This possibility only occurs when there are at least three records in `data_in`, as the procedure reduces to
%             the standard Procrustes algorithm, which finds the consensus when there are only two records, is deterministic other than does rotational ambiguity.
%
%     - The swarm option (opts_knit.if_swarm=1) can be used to automatically step through several types of initialization methods, and choose the best.
%     - If allow_scale=1, then the optimal solution will depend not only on
%     the data points that overlap, but also on the data points that are unique to individual components, since these contribute to determining tthe rescaling on each iteration
% 
% Note: Note regarding replotting a previous analysis
%     - To replot a previous calculation with additional customization or to make a composite figure, proceed as follows:
%
%          - `data_in` should be equal to that used in the previous calculation.
%          - aux.knit_stats should be equal to aux_out.knit_stats from the previous calculation
%          - aux.knit_stats_setup should be equal to aux_out.knit_stats_setup from the previous calculation
%
%     - The following modifications are allowed in fields of knit_stats_setup:
%
%         - dataset_labels (cell array of char): dataset labels; default is data_in.sets{:}.label
%         - stimulus_labels (cell array of char): stimulus labbels; default is data_out.sas{1}.typenames
%         - shuff_quantiles (float 1-d array): quantiles to show; default is 0.01 0.05 0.5 0.95 0.99
%         - fig_handle (handle): figure handle to plot into; figure will be created if not supplied
%         - row (int): row to plot into; default is 1
%         - nrows (int): number of rows in the figure; default is row
%
%     -  No further calculations are done
%     -  On return, data_out will be empty, and aux_out.fig_handle will be the figure handle
%     -  In creating a composite figure, rows should be plotted in order from top to bottom, as plotting the bottom row triggers an equalization of the color scale. See `rs_knit_coordsets_demo` for an example.
%
% Note: Note regarding swarm optimization
%     - This is enabled by setting opts_knit.if_swarm=1.
%     - Preliminary runs are made with several classes of initial guesses, and stopping parameters based on max_niters and max_rmstol in opts_knit.swarm
%     - The choice that leads to the smallest rms deviation of the aligned coordinates is recorded in aux.details{idim}.swarm, and used for a definitive run, using the stopping parameters in opts_knit
%     - The classes are
%
%         - init_components: the original component datasets are used; variants choose each component dataset in succession as the initial guess, advancing to the next dataset if a stimulus is missing
%         - init_pca: principal components of the datasets are used, with options (mean subtraction, rotation) as described in the note regarding the Procrustes alignment algorithm; variants consist of computing the principal components from the coordinates of stimuli present in increasing numbers of the datasets
%         - init_random: Gaussian random varibles are used as the initial guess; the number of variants is given by opts_knit.swarm.nrand; default is 10; these random numbers do not affect the random numbers used for permutation testing 
%
%     - Results from each class are recorded in aux.details{idim}.(class_name), including
%
%         - counts: the number of variants for each class
%         - rms_dev_totals: the rms deviation for each example of the class
%         - range: the range of rms deviations for each class
%         - itry_min: the optimal choice within each class
%         - opts: the options usef for the variant with the lowest rms deviation
%
% Note: Note regarding rays
%     - The `ray structure` describes relationships among the simulus coordinates: 
%     `rays`, i.e., sets of stimuli that lie along an axis or a ray from the origin,
%     `rings`, stimuli that lie at an approximately equal distance from the origin, and nearest neighbors.
%     It is only created if there is a valid set of stimulus coordinates.  
%
% See also:
%   RS_ALIGN_COORDSETS, RS_AUX_CUSTOMIZE, RS_CHECK_COORDSETS, RS_FINDRAYS,
%   PSG_ALIGN_COORDSETS, PSG_KNIT_STATS,
%   PSG_REMNAN_COORDSETS, PSG_COORD_PIPE_UTIL, OVERLAP_HEURISTICS, PROCRUSTES_CONSENSUS, PSG_ALIGN_STATS_PLOT.
%
if (nargin<=1)
    aux=struct;
end
aux=filldefault(aux,'opts_knit',struct);
aux.opts_knit=filldefault(aux.opts_knit,'if_log',1);
aux.opts_knit=filldefault(aux.opts_knit,'allow_reflection',1);
aux.opts_knit=filldefault(aux.opts_knit,'allow_offset',1);
aux.opts_knit=filldefault(aux.opts_knit,'allow_scale',0);
aux.opts_knit=filldefault(aux.opts_knit,'if_normscale',aux.opts_knit.allow_scale);
aux.opts_knit=filldefault(aux.opts_knit,'if_stats',0);
aux.opts_knit=filldefault(aux.opts_knit,'if_plot',aux.opts_knit.if_stats);
aux.opts_knit=filldefault(aux.opts_knit,'if_pca',0);
aux.opts_knit=filldefault(aux.opts_knit,'max_niters',1000);
aux.opts_knit=filldefault(aux.opts_knit,'max_rmstol',10^-5);
aux.opts_knit=filldefault(aux.opts_knit,'pcon_init_method',0);
aux.opts_knit=filldefault(aux.opts_knit,'keep_details',0);
aux.opts_knit=filldefault(aux.opts_knit,'pcon_initial_guess',[]);
aux.opts_knit=filldefault(aux.opts_knit,'pcon_alignment',aux.opts_knit.pcon_initial_guess);
aux.opts_knit=filldefault(aux.opts_knit,'if_frozen',1);
aux.opts_knit=filldefault(aux.opts_knit,'if_remove_path_label',1);
aux.opts_knit=filldefault(aux.opts_knit,'if_dim_heuristics',0);
aux.opts_knit=filldefault(aux.opts_knit,'if_swarm',0);
%
aux=filldefault(aux,'opts_check',struct);
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
if aux.opts_knit.if_stats
    aux.opts_knit=filldefault(aux.opts_knit,'nshuffs',500);
else
    aux.opts_knit.nshuffs=0;
end
aux.opts_knit=filldefault(aux.opts_knit,'shuff_quantiles',[0.01 0.05 0.5 0.95 0.99]);
%
aux=filldefault(aux,'opts_pca',struct);
aux.opts_pca=filldefault(aux.opts_pca,'if_log',0);
aux.opts_pca=filldefault(aux.opts_pca,'nd_max',Inf);
%
aux=filldefault(aux,'opts_rays',struct);
%
aux=filldefault(aux,'opts_align',struct);
%
aux=rs_aux_customize(aux,'rs_knit_coordsets');
%
data_out=struct;
aux_out=struct;
%
set_knit_strings={'paradigm_name','subj_id','subj_id_short','extra','label_long','label'}; %fields to be concatenated in knitted metadata
%
%check consistency and get available stimuli, dimensions, typenames
%
check=rs_check_coordsets(data_in,aux.opts_check);
%
aux_out.warnings=check.warnings;
aux_out.warn_bad=check.warn_bad;
%
% replot mode
%
if isfield(aux,'knit_stats') & isfield(aux,'knit_stats_setup')
    knit_stats_setup_use=aux.knit_stats_setup;
    if isfield(knit_stats_setup_use,'fig_handle') %psg_knit_stats_plot expects figure handle in figh
        knit_stats_setup_use.figh=knit_stats_setup_use.fig_handle;
    end
    aux_out.fig_handle=psg_knit_stats_plot(aux.knit_stats,knit_stats_setup_use);
    return
end
%
nsets=check.nsets;
nstims_each=check.nstims_each;
dim_list_each=check.dim_list_each;
dim_list_union=check.dim_list_union;
dim_list_inter=check.dim_list_inter;
typenames_each=check.typenames_each;
typenames_union=check.typenames_union;
typenames_inter=check.typenames_inter;
%
if min(nstims_each)~=max(nstims_each)
    disp('cannot proceed');
    disp(aux_out.warnings);
    return
end
%
%inspect input data to see where data are missing
%note that a NaN can indicate that stimulus was present and response
%was missing, OR, that the stimulus was not presented
%
nstims_all=min(nstims_each);
coords_isnan=zeros(nstims_all,nsets);
for iset=1:nsets
    for kd=dim_list_each{iset}
        coords_isnan(:,iset)=or(coords_isnan(:,iset),any(isnan(data_in.ds{iset}{kd}),2)); %if any coordinate is missing for any dimension, here we consider it missing
    end
    if aux.opts_knit.if_log
        disp(sprintf(' number of stimuli missing in dataset %3.0f: %4.0f',iset,sum(coords_isnan(:,iset),1)));
    end
end
aux_out.coords_havedata=1-coords_isnan;
if aux.opts_knit.if_log
    disp('data table')
    disp(aux_out.coords_havedata'*aux_out.coords_havedata)
end
%
%if aux.sa_pooled is present, use it, otherwise, re create
if (isfield(aux,'sa_pooled') & isfield(aux,'data_align'))
    if (aux.opts_knit.if_log)
        disp('sa_pooled and data_align are supplied.');
    end
    sa_pooled=aux.sa_pooled;
    data_align=aux.data_align;
    opts_align_used=aux.opts_align;
else
    if (aux.opts_knit.if_log)
        disp('sa_pooled and data_align will be created.');
    end
    %redo the alignment, but first remove the nans in the aligned file; this would confuse if realigned
    [sets_nonan,ds_nonan,sas_nonan]=psg_remnan_coordsets(data_in.sets,data_in.ds,data_in.sas,[],setfield(struct,'if_log',aux.opts_knit.if_log));
    data_in_nonan=struct;
    data_in_nonan.ds=ds_nonan;
    data_in_nonan.sas=sas_nonan;
    data_in_nonan.sets=sets_nonan;
    aux2=aux;
    aux2.opts_align.if_log=aux.opts_knit.if_log;
    [data_align,aux_align]=rs_align_coordsets(data_in_nonan,aux2);
    sa_pooled=aux_align.sa_pooled;
    opts_align_used=aux_align.opts_align;
end
if any(all(coords_isnan,2))
   wmsg='at least one stimulus is not present in any dataset';
   aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
   disp('table of missing data')
    for istim=1:nstims_all
        disp(cat(2,sprintf(' %12s',sa_pooled.typenames{istim}),sprintf(' %2.0f',coords_isnan(istim,:))));
    end
end
if length(intersect(sa_pooled.typenames,typenames_union))~=length(union(sa_pooled.typenames,typenames_union))
    wmsg=sprintf('pooled typenames are incompatible with type names from individual datasets');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
    disp('discrepancies')
    disp(setdiff(typenames_union,sa_pooled.typenames));
end
%
%set up dimension defaults
%
dim_list_all=dim_list_inter;
aux.opts_knit=filldefault(aux.opts_knit,'dim_max_in',max(dim_list_all));
aux.opts_knit=filldefault(aux.opts_knit,'dim_list_in',[1:aux.opts_knit.dim_max_in]);
aux.opts_knit=filldefault(aux.opts_knit,'dim_aug',0);
aux.opts_knit=filldefault(aux.opts_knit,'dim_list_out',aux.opts_knit.dim_aug+aux.opts_knit.dim_list_in);
if length(aux.opts_knit.dim_list_in)~=length(aux.opts_knit.dim_list_out)
    wmsg=sprintf('dim_list_in and dim_list_out have different lengths');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
else
    if_aug=any(aux.opts_knit.dim_list_out>aux.opts_knit.dim_list_in);
    aux.opts_knit=filldefault(aux.opts_knit,'if_initpca_rot',1-if_aug);
    %
    if any(aux.opts_knit.dim_list_out<aux.opts_knit.dim_list_in)
        wmsg=sprintf('dim_list_in values cannot be more than dim_list_out values');
        aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
    end
end
if length(aux.opts_knit.dim_list_in)~=length(unique(aux.opts_knit.dim_list_in))
    wmsg=sprintf('dim_list_in values are not unique');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
end
if length(aux.opts_knit.dim_list_out)~=length(unique(aux.opts_knit.dim_list_out))
    wmsg=sprintf('dim_list_out values are not unique');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
end
%
aux_out.opts_check=aux.opts_check;
if aux_out.warn_bad==0
    %
    %do dimension heuristics if more than one dataset
    %
    overlaps=1-coords_isnan;
    if nsets>1
        h=overlap_heuristics(overlaps);
        if h.dmax<max(aux.opts_knit.dim_list_out)
            if_hbad=1;
        else
            if_hbad=0;
        end
    else
        if_hbad=0; %dimension heuristics always ok if only one set
    end
    if aux.opts_knit.if_dim_heuristics==1 | if_hbad==1
        if nsets>1
            disp(sprintf('dimension limit estimated at %3.0f (limit due to un-duplicated stimuli: %3.0f, limit due to number of overlapping distances: %3.0f)',...
                h.dmax,h.dmax_free,h.dmax_constraints));
            if if_hbad
                wmsg=sprintf('number of overlaps between datasets may be insufficient for dimensions >= %2.0f; dimensions %2.0f to %2.0f requested',...
                    h.dmax,min(aux.opts_knit.dim_list_out),max(aux.opts_knit.dim_list_out));
                aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
            end
        end
        disp('table of available data')
        for istim=1:nstims_all
            disp(cat(2,sprintf(' %12s',sa_pooled.typenames{istim}),sprintf(' %2.0f',overlaps(istim,:))));
        end
    end
    %
    %process
    %
    typenames_all=typenames_inter; %because stimuli are required to be the same across datasets
    dim_list_in=aux.opts_knit.dim_list_in;
    dim_list_out=aux.opts_knit.dim_list_out;
    %
    if aux.opts_knit.if_log
        disp(sprintf('knitting %3.0f stimuli across %3.0f datasets, dimensions %s',nstims_all,nsets,sprintf(' %2.0f',dim_list_in))); 
        disp(sprintf('  allow reflection: %1.0f, allow offset: %1.0f, allow scale: %1.0f, normalize scale: %1.0f, rotate to pcs: %1.0f',...
            aux.opts_knit.allow_reflection,aux.opts_knit.allow_offset,aux.opts_knit.allow_scale,aux.opts_knit.if_normscale,aux.opts_knit.if_pca));
    end
    if aux.opts_knit.if_pca
        c2p_string='-pc';
    else
        c2p_string='';
    end
    if ischar(aux.opts_knit.pcon_init_method)
        if strcmp(aux.opts_knit.pcon_init_method,'specify')
            aux.opts_knit.initialize_set=0; %opts_knit.pcon_initial_guess and opts_knit.pcon_alignment will be used
        else
            wmsg='initialization method not recognized; default used';
            aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
            aux.opts_knit.pcon_init_method=0;
            aux.opts_knit.initialize_set='pca';
        end
    else
        if aux.opts_knit.pcon_init_method>0
            aux.opts_knit.initialize_set=aux.opts_knit.pcon_init_method;
        elseif aux.opts_knit.pcon_init_method==0
            aux.opts_knit.initialize_set='pca';
        elseif aux.opts_knit.pcon_init_method==-1
            aux.opts_knit.initialize_set='pca_center';
        else
            aux.opts_knit.initialize_set='pca_nocenter';
        end
    end
    %
    %do a consensus 
    %
    opts_pcon=aux.opts_knit;
    [ra,warnings,details]=psg_knit_stats(data_align.ds,data_align.sas,dim_list_in,dim_list_out,opts_pcon);
    if ~isempty(warnings)
        wmsg=strvcat(wmsg,warnings);
        warn_leadin=getfield(getfield(rs_aux_customize(struct()),'overall'),'warn_leadin');
        for k=1:size(warnings,1)
            disp(cat(2,warn_leadin,warnings(k,:)));
        end
    end
    ds_knitted=ra.ds_knitted;
    ds_components=ra.ds_components;
    opts_pcon_used=ra.opts_pcon_eachdim'; %make a column for consistency 
    details=details'; %make a column for consistency
    %
    %implement PCA rotation if requested:  note that this is applied both
    %to consensus and components in output, but not in knit_stats.components
    %
    if aux.opts_knit.if_pca
        ts_pca=cell(1,max(dim_list_in));
        for dptr=1:length(dim_list_out)
            ip=dim_list_in(dptr);
            ip_out=dim_list_out(dptr);
            knitted_centroid=mean(ds_knitted{ip_out},1,'omitnan');
            [ds_knitted{ip_out},recon_coords,var_ex,var_tot,coord_maxdiff,opts_used_pca]=psg_pcaoffset(ds_knitted{ip_out},knitted_centroid,aux.opts_pca);
    %        qu=opts_used_pca.qu;
    %        qs=opts_used_pca.qs;
            v=opts_used_pca.qv;
    %       coords=u*s*v', and recon_coords= u*s, with v'*v=I, so recon_coords=coords*v
            knitted_centroid_rep=repmat(knitted_centroid,nstims_all,1);
            for iset=1:nsets
                ds_components{iset}{1,ip_out}=knitted_centroid_rep+(ds_components{iset}{1,ip_out}-knitted_centroid_rep)*v(1:ip_out,:);
                %determine transformation to consensus followed by pca
                ts_pca{ip}{iset}.scaling=ra.ts{ip}{iset}.scaling;
                ts_pca{ip}{iset}.orthog=ra.ts{ip}{iset}.orthog*v(1:ip_out,:);
                ts_pca{ip}{iset}.translation=knitted_centroid_rep(1,:)+(ra.ts{ip}{iset}.translation-knitted_centroid_rep(1,:))*v(1:ip_out,:);
            end
        end %dptr
        aux_out.ts_pca=ts_pca;
    end
    %
    %if statistics, keep them
    %
    if aux.opts_knit.if_stats
        knit_stats_setup.nsets=nsets;
        knit_stats_setup.dim_list_in_max=max(dim_list_in);
        knit_stats_setup.dim_list_in=dim_list_in;
        knit_stats_setup.dim_list_out=dim_list_out;
        knit_stats_setup.dataset_labels=cell(1,nsets);
        for iset=1:nsets
            ds_label=data_in.sets{iset}.label;
            if aux.opts_knit.if_remove_path_label
                path_end=max(max(union(find(ds_label=='\'),find(ds_label=='/'))),0);
                if ~isempty(path_end)
                    ds_label=ds_label(1+path_end:end);
                    ds_label=strrep(ds_label,'.mat','');
                end
            end
            knit_stats_setup.dataset_labels{iset}=ds_label;
        end
        knit_stats_setup.stimulus_labels=typenames_all;
        knit_stats_setup.nshuffs=aux.opts_knit.nshuffs;
        knit_stats_setup.shuff_quantiles=aux.opts_knit.shuff_quantiles;
        knit_stats_setup.nstims=nstims_all;
        %
        aux_out.knit_stats=ra;
        aux_out.knit_stats_setup=knit_stats_setup;
        if aux.opts_knit.if_plot
            knit_stats_setup_use=aux_out.knit_stats_setup;
            if isfield(knit_stats_setup_use,'fig_handle')
                knit_stats_setup_use.figh=knit_stats_setup_use.fig_handle; %psg_knit_stats_plot expects figure handle in figh
            end
            aux_out.fig_handle=psg_knit_stats_plot(aux_out.knit_stats,knit_stats_setup_use);
        end
    end
    sas_knitted=sa_pooled;
    %
    %knitted set structure
    sets_knitted=struct;
    sets_knitted.nstims=nstims_all;
    sets_knitted.dim_list=dim_list_out;
    for ifn=1:length(set_knit_strings)
        fn=set_knit_strings{ifn};
        sets_knitted.(fn)=''; % was []
        counts=0;
        for iset=1:nsets
            if isfield(data_in.sets{iset},fn)
                sets_knitted.(fn)=cat(2,sets_knitted.(fn),char(data_in.sets{iset}.(fn)),'+');
                counts=counts+1;
            end
        end
        if counts>=1 %remove trailing +
            sets_knitted.(fn)=sets_knitted.(fn)(1:end-1);
        end
    end
    pipeline_opts=struct;
    pipeline_opts.opts_knit=aux.opts_knit;
    pipeline_opts.opts_pcon=opts_pcon_used;
    sets_knitted.pipeline=psg_coord_pipe_util('knit',pipeline_opts,[],[],data_in.sets);
    %find rays
    [rays,wmsg,opts_rays_used]=rs_findrays(sas_knitted,sets_knitted.label,aux.opts_rays);
    if ~isempty(wmsg)
        aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
    end
    %
    %dim list and pipeline for component sets
    for iset=1:nsets
        data_in.sets{iset}.dim_list=dim_list_out;
        data_in.sets{iset}.pipeline=psg_coord_pipe_util('knit',pipeline_opts,data_in.sets{iset},[],data_in.sets);       
    end
    %propagate paradigm type (must match if sets{:}.type='data', checked in rs_align_coordsets
    paradigm_type=[];
    for iset=1:nsets
        if strcmp(data_in.sets{iset}.type,'data')
            if isfield(data_in.sets{iset},'paradigm_type')
                if isempty(paradigm_type) 
                    paradigm_type=data_in.sets{iset}.paradigm_type;
                end
            end
        end
    end
    data_out.ds{1}=ds_knitted;
    data_out.sas{1}=sas_knitted;
    data_out.sets{1}=sets_knitted;
    data_out.sets{1}.type=data_in.sets{1}.type;
    if ~isempty(paradigm_type)
        data_out.sets{1}.paradigm_type=paradigm_type;
    end
    %
    aux_out.rayss{1}=rays;
    aux_out.opts_rays{1}=opts_rays_used;
    aux_out.opts_knit=aux.opts_knit;
    aux_out.opts_pcon=opts_pcon_used;
    aux_out.opts_pca=aux.opts_pca;
    aux_out.opts_align=opts_align_used;
    %
    aux_out.components.ds=ds_components;
    aux_out.components.sas=data_align.sas;
    aux_out.components.sets=data_in.sets;
    %
    if aux.opts_knit.keep_details
        aux_out.details=details;
    end
else
    disp('cannot proceed');
    disp(aux_out.warnings);
end
return
end

`rs_plot_style(coords, plotstyle, opts)`

[handles,plotstyles_used,opts_used]=rs_plot_style(coords,plotstyle,opts) plots a set of points into the current axis

The data may be either 2D or 3D. Marker type, marker size, line style, line thickness, and alpha blending, may be specified. Possible conflicts or unsupported properties are handled.

Input arguments:

coords (float 2-D array) –

data to plot, 2 columns for a 2D plot, 3 columns for a 3D plot
plotstyle (struct) –
specification of plotting style, may be omitted, with fields
- marker (char) – marker type; one of {'.','o','x','+','*','s','d','v','^','<','>','p','h'}; default is '.'
- markersize (int) – marker size; default is 6
- linestyle (char) – line style; one of {'-',':','-.','--',','none'}; default is 'none'
- linewidth (int) – width of lines and marker edges; default is 1
- color (char or float 1-D array) – color of points, lines, and marker edges, can be any Matlab color specifier or an (r,g,b) triple; default is 'k' (black)
- color_fill (char or fload 1-D array) – color of marker interior if marker is filled; default is plotstyle.color;
- filled (int) – 1 to filled marker, 0 for unfilled; default is 0
- alpha (float) – alpha-blending (transparency); defaults to 1 (opaque)
opts (struct) –
how to resolve conflicts (see note below regarding graphic hints), may be omitted, with fields
- if_alpha_color_line (int) – 1 if line color property allows for alpha as fourth component, 0 if not, -1 to determine from hints; default is -1
- if_alpha_color_line_marker (int) – 1 if marker edge color property allows for alpha as fourth component, 0 if not, -1 to determine from hints; default is -1
- if_alpha_scatter_marker_edge (int) – 1 if MarkerEdgeAlpha is a property of Scatter, 0 if not, -1 to determine from hints; default is -1
- if_alpha_scatter_marker_face (int) – 1 if MarkerFaceAlpha is a property of Scatter, 0 if not, -1 to determine from hints; default is -1

Output arguments:

handles ( struct ) –
structure of handles to the plot and components, with fields
- line (struct) – handle of the graphics line object used for the line connecting points; may be empty
- markers (struct) – handle of the graphics object for the plotted points, may be a line or scatter object
- scatter (struct) – handle of the graphics scatter object for the plotted points; may be empty
- legend (struct) – handle to graphics object (line or scatter) that can be used for a legend; will be either handles.line or handles.scatter and will not be empty
plotstyles_used ( struct ) –

plot styles with defaults filled in
opts_used ( struct ) –

opts, with defaults filled in; see note below regarding graphic hints

General notes

Plotting will be into current axis if available; otherwise a new axis will be created.
Axis hold state will be 'on' after plotting.

Note regarding graphic hints

Capabilities for alpha-blending may be version-dependent, and should be indicated during installation by customizing rs_graphic_hints
An entry of -1 (default) in 'opts' uses rs_graphic_hints to determine the capability.
If rs_graphic_hints is absent, the present, and if not, by attempting to set an alpha property
At run-time, these hints may be overrridden by a 1 (capability present) or a 0 (capability absent) in a field of 'opts'.
If an attempt to use transparency fails, then the corresponding field of 'opts_used' is set to 0, and a essage is generated in opts_used.msgs

Source code in src/rs_plot_style.m

function [handles,plotstyles_used,opts_used]=rs_plot_style(coords,plotstyle,opts)
% [handles,plotstyles_used,opts_used]=rs_plot_style(coords,plotstyle,opts)
% plots a set of points into the current axis
% 
% The data may be either 2D or 3D. Marker type, marker size,
% line style, line thickness, and alpha blending, may be specified.  Possible conflicts or unsupported properties are handled.
%
% Args:
%   coords (float 2-D array): data to plot, 2 columns for a 2D plot, 3 columns for a 3D plot
%
%   plotstyle (struct): specification of plotting style, may be omitted, with fields
%
%      - marker (char): marker type; one of {'.','o','x','+','*','s','d','v','^','<','>','p','h'}; default is '.'
%      - markersize (int): marker size; default is 6
%      - linestyle (char): line style; one of {'-',':','-.','--',','none'}; default is 'none'
%      - linewidth (int): width of lines and marker edges; default is 1
%      - color (char or float 1-D array): color of points, lines, and marker edges, can be any Matlab color specifier or an (r,g,b) triple; default is 'k' (black)
%      - color_fill (char or fload 1-D array): color of marker interior if marker is filled; default is plotstyle.color;
%      - filled (int): 1 to filled marker, 0 for unfilled; default is 0
%      - alpha (float): alpha-blending (transparency); defaults to 1 (opaque)
%
%   opts (struct): how to resolve conflicts (see note below regarding graphic hints), may be omitted, with fields
%
%      - if_alpha_color_line (int): 1 if line color property allows for alpha as fourth component, 0 if not, -1 to determine from hints; default is -1 
%      - if_alpha_color_line_marker (int): 1 if marker edge color property allows for alpha as fourth component, 0 if not, -1 to determine from hints; default is -1 
%      - if_alpha_scatter_marker_edge (int): 1 if MarkerEdgeAlpha is a property of Scatter, 0 if not, -1 to determine from hints; default is -1
%      - if_alpha_scatter_marker_face (int): 1 if MarkerFaceAlpha is a property of Scatter, 0 if not, -1 to determine from hints; default is -1
%
%
% Returns:
%   handles (struct): structure of handles to the plot and components, with fields
%
%     - line (struct): handle of the graphics line object used for the line connecting points; may be empty
%     - markers (struct): handle of the graphics object for the plotted points, may be a line or scatter object
%     - scatter (struct): handle of the graphics scatter object for the plotted points; may be empty
%     - legend (struct): handle to graphics object (line or scatter) that can be used for a legend; will be either handles.line or handles.scatter and will not be empty
% 
%   plotstyles_used (struct): plot styles with defaults filled in
% 
%   opts_used (struct): opts, with defaults filled in; see note below regarding graphic hints
% 
% Note: General notes
%    - Plotting will be into current axis if available; otherwise a new axis will be created.
%    - Axis hold state will be 'on' after plotting.
%
% Note: Note regarding graphic hints
%    - Capabilities for alpha-blending may be version-dependent, and should be indicated during installation by customizing `rs_graphic_hints`
%    - An entry of -1 (default) in 'opts' uses `rs_graphic_hints` to determine the capability.
%    - If `rs_graphic_hints` is absent, the present, and if not, by attempting to set an alpha property
%    - At run-time, these hints may be overrridden by a 1 (capability present) or a 0 (capability absent) in a field of 'opts'.
%    - If an attempt to use transparency fails, then the corresponding field of 'opts_used' is set to 0, and a essage is generated in opts_used.msgs
% 
%   See also:  RS_PLOT_STYLE_TEST, RS_GRAPHIC_HINTS.
%
if (nargin<=1)
    plotstyle=struct;
end
if (nargin<=2)
    opts=struct;
end
if exist('rs_graphic_hints','file')
    rs_graphic_hint_def=rs_graphic_hints();
end
if ~exist('rs_graphic_hints_def')
    rs_graphic_hints_def=struct;
end
opts=rs_plot_style_sethint(opts,'if_alpha_color_line',rs_graphic_hints_def); %line color allows for alpha
opts=rs_plot_style_sethint(opts,'if_alpha_color_line_marker',rs_graphic_hints_def); %MarkerEdgeColor and MarkerFaceColor on a line allow for alpha
opts=rs_plot_style_sethint(opts,'if_alpha_scatter_marker_edge',rs_graphic_hints_def); %MarkerEdgeAlpha is a property of Scatter
opts=rs_plot_style_sethint(opts,'if_alpha_scatter_marker_face',rs_graphic_hints_def); %MarkerEdgeAlpha is a property of Scatter
% 
plotstyle_def=struct;
plotstyle_def.marker='.';
plotstyle_def.markersize=6;
plotstyle_def.filled=0;
%
plotstyle=filldefault(plotstyle,'marker',plotstyle_def.marker);
plotstyle=filldefault(plotstyle,'markersize',plotstyle_def.markersize);
plotstyle=filldefault(plotstyle,'linestyle','none');
plotstyle=filldefault(plotstyle,'linewidth',1);
plotstyle=filldefault(plotstyle,'color','k');
plotstyle=filldefault(plotstyle,'color_fill',plotstyle.color);
plotstyle=filldefault(plotstyle,'filled',plotstyle_def.filled);
plotstyle=filldefault(plotstyle,'alpha',1);
plotstyles_used=plotstyle;
handles=[];
opts_used=opts;
opts_used.msgs=[];
%
nds=size(coords,2);
if ~ismember(nds,[0 2 3])
    return
end
%
handles.legend=[];
handles.line=[];
handles.markers=[];
handles.scatter=[];
%
if plotstyle.alpha==1
    hp=rs_plot_style_do(coords,'Line');
    set(hp,'Color',plotstyle.color);
    set(hp,'Marker',plotstyle.marker);
    set(hp,'MarkerSize',plotstyle.markersize);
    set(hp,'MarkerEdgeColor',plotstyle.color);
    if plotstyle.filled
        set(hp,'MarkerFaceColor',plotstyle.color_fill);
    end
    set(hp,'LineStyle',plotstyle.linestyle);
    set(hp,'LineWidth',plotstyle.linewidth);
    handles=rs_plot_style_sethandles(handles,hp,{'legend','line','markers'});
    return
%now deal with alpha
else
    %
    if_alpha_marker=~strcmp(plotstyle.marker,'none');
    if_alpha_line=~strcmp(plotstyle.linestyle,'none');
    %
    %alpha for line but not marker
    if if_alpha_line & ~if_alpha_marker
        [hp,opts_used]=rs_plot_style_line(coords,plotstyle,plotstyle_def,opts_used);
        handles=rs_plot_style_sethandles(handles,hp,{'legend','line'});
    end 
    %
    %alpha requested for marker but not line
    if ~if_alpha_line & if_alpha_marker
        [hp,opts_used]=rs_plot_style_scatter(coords,plotstyle,plotstyle_def,opts_used);
        handles=rs_plot_style_sethandles(handles,hp,{'legend','scatter','markers'});
    end %alpha requested for marker but not line
    %
    %alpha requested for line and marker
    if if_alpha_line & if_alpha_marker
        %first plot line with no markers
        [hp,opts_used]=rs_plot_style_line(coords,setfield(plotstyle,'marker','none'),plotstyle_def,opts_used);
        handles=rs_plot_style_sethandles(handles,hp,{'line'});
        %then plot markers without line, and use this for legend handle
        [hp,opts_used]=rs_plot_style_scatter(coords,plotstyle,plotstyle_def,opts_used);
        handles=rs_plot_style_sethandles(handles,hp,{'legend','scatter','markers'});
    end
end
return
end

`rs_read_choicedata(fullname, aux)`

[data_comp,aux_out]=rs_read_choicedata(fullname,aux) reads a single choice file and validates entries

The choice file may be triadic or tetradic, as determined by the number of columns in the choices variable of the data file: 5 columns if triadic, 6 columns if tetradic Each row of the variable 'responses' holds responses from one or more presentations of a single comparison. Comparisons may be repeated in other rows, and a comparison may also be repeated with a separate indexing (e.g., s1 and s2 may be reversed).

Important note: The standard convention in the choice file is to count the number of times the first comparison is judged to be more different than the second; in 'data_comp', the opposite convention is used. This conversion is made automatically and logged.

Input arguments:

fullname (char or singleton char array) –

full file name with path; if empty, it will be requested interactively. File names must contain the string '_coords'. See note below regarding setup files.
aux (struct) –
a structure, can be omitted, with fields
- opts_read (struct) – options for reading, can be omitted, with fields
  - if_gui (int) – 1 to use graphical interface to request data file name if 'fullname' is empty, 0 to use console; default is 1; see note below regarding customization
  - if_log (int) – 1 to log progress, 0 to omit; default is 1; see note below regarding customization
  - data_fullname_def (char) – prompt for data file if 'fullname' is empty; see note below regarding customization
  - ui_filter (char) – file name filter that appears in graphical user interface when if_gui=1; defaults to '_choices'; see note below regarding customization
  - if_consolidate (int) – 1 to consolidate equivalent triadic and tetradic judgments into a single line, 0 does not; default is 1
- opts_check (struct) – options for consistency checking, with field
  - if_warn (int) – 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1

Output arguments:

data_comp ( int 2-D array ) –
Choice data, with each row containing the data from a single kind of comparison
- first 3 or 4 columns: indexes into the stimuli used for the comparison
  - triadic: col 1 is reference, col 2 is s1, col 3 is s2, comparisons are (ref,s1) and (ref,s2)
  - tetradic: cols 1-4 are s1-s4, comparisons are (s1,s2) and (s3,s4)
- next column: number of times the first difference was judged more similar than the second difference
- final column: number of times the comparison was made
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- typenames (cell array) – stimulus labels, corresponding to the indices used in the first 3 or 4 columns of choices. The order is unchanged from the order in the data file.
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_read (cell array of struct) – opts_read{1} is aux.opts_read, with defaults filled in
- opts_check (struct) – aux.opts_check, with defaults filled in

Note regarding customization

The defaults for the following parameters can be set by editing the line containing generic.opts_read.[param_name] in rs_aux_defaults_define, running it once, and saving the workspace as rs_aux_defaults.mat.
- if_gui
- if_log
- data_fullname_def ('_coords' will be replaced by '_choices' at run time)
- coord_string
- type_class_aux

See also: rs_aux_customize, rs_check_coordsets, PSG_READ_COORDDATA.

Source code in src/rs_read_choicedata.m

function [data_comp,aux_out]=rs_read_choicedata(fullname,aux)
% [data_comp,aux_out]=rs_read_choicedata(fullname,aux) reads a single `choice file` and validates entries
%
% The `choice file` may be triadic or tetradic, as determined by the number of columns in the choices variable of the data file:  5 columns if triadic, 6 columns if tetradic
% Each row of the variable 'responses' holds responses from one or more presentations of a single comparison.  Comparisons may be repeated in other rows, and a comparison may also be repeated with a separate indexing (e.g., s1 and s2 may be reversed).
%
% __Important note__: The standard convention in the `choice file` is to count the number of times the first comparison is judged to be more different than the second; in 'data_comp', the __opposite__ convention is used.
% This conversion is made automatically and logged.
%
% Args:
%   fullname (char or singleton char array): full file name with path; if empty, it will be requested interactively. File names must contain the string '_coords'. See note below regarding setup files.
%
%   aux (struct): a structure, can be omitted, with fields 
%
%     - opts_read (struct): options for reading, can be omitted, with fields
%
%         - if_gui (int): 1 to use graphical interface to request data file name if 'fullname' is empty, 0 to use console; default is 1; see note below regarding customization
%         - if_log (int): 1 to log progress, 0 to omit; default is 1; see note below regarding customization
%         - data_fullname_def (char): prompt for data file if 'fullname' is empty; see note below regarding customization
%         - ui_filter (char): file name filter that appears in graphical user interface when if_gui=1; defaults to '*_choices*'; see note below regarding customization
%         - if_consolidate (int): 1 to consolidate equivalent triadic and tetradic judgments into a single line, 0 does not; default is 1
%
%     - opts_check (struct): options for consistency checking, with field
%
%         - if_warn (int): 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
%
% Returns:
%   data_comp (int 2-D array): Choice data, with each row containing the data from a single kind of comparison
%
%      - first 3 or 4 columns: indexes into the stimuli used for the comparison
% 
%          - triadic: col 1 is reference, col 2 is s1, col 3 is s2, comparisons are (ref,s1) and (ref,s2)
%          - tetradic: cols 1-4 are s1-s4, comparisons are (s1,s2) and (s3,s4)
%
%      - next column: number of times the first difference was judged __more similar than__ the second difference
%      - final column: number of times the comparison was made
%
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - typenames (cell array):  stimulus labels, corresponding to the indices used in the first 3 or 4 columns of choices.  The order is unchanged from the order in the data file.
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_read (cell array of struct): opts_read{1} is aux.opts_read, with defaults filled in
%     - opts_check (struct): aux.opts_check, with defaults filled in
%
% Note: Note regarding customization
%     - The defaults for the following parameters can be set by editing the line containing generic.opts_read.[param_name] in  `rs_aux_defaults_define`, running it once, and saving the workspace as rs_aux_defaults.mat.
%
%         - if_gui
%         - if_log
%         - data_fullname_def ('_coords' will be replaced by '_choices' at run time)
%         - coord_string
%         - type_class_aux
%
%  See also: RS_AUX_CUSTOMIZE, RS_CHECK_COORDSETS, PSG_READ_COORDDATA.
%
valid_choice_types={'triadic','tetradic'};
%
if (nargin<=1)
    aux=struct;
end
aux=filldefault(aux,'opts_read',struct);
aux.opts_read=filldefault(aux.opts_read,'if_consolidate',1);
%
aux=filldefault(aux,'opts_check',struct);
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
aux.opts_read.ui_filter='*_choices*';
aux=rs_aux_customize(aux,'rs_read_choicedata');
%
aux_out=struct;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
if iscell(fullname)
    fullname=fullname{1};
end
%
if isempty(fullname)
    if aux.opts_read.if_gui
        if_manual=0;
        ui_prompt='Select a choice file';
        ui_filter=aux.opts_read.ui_filter;       
        while (if_manual==0 & isempty(fullname))
            [filename_short,pathname]=uigetfile(ui_filter,ui_prompt,'Multiselect','off');
            if  (isequal(filename_short,0) | isequal(pathname,0)) %use Matlab's suggested way to detect cancel
                if_manual=getinp('1 to return to selection from console','d',[0 1]);
            else
                fullname=cat(2,pathname,filename_short);
            end
        end
    end
end
aux.opts_read.nometa=1;
aux.opts_read.sign_check_mode=0; %look for sign (< or >) in responses_colnames, and ask if it is not found
aux.opts_read.data_fullname_def=strrep(aux.opts_read.data_fullname_def,'_coords','_choices');
aux.opts_read.permutes=[]; %so that psg_read_coorddata will not attempt to permute rays
[data_comp,sa,opts_read_used]=psg_read_choicedata(fullname,[],aux.opts_read);
%
aux_out.typenames=sa.typenames;
aux_out.opts_read=opts_read_used;
aux_out.opts_check=aux.opts_check;
%
ncols=size(data_comp,2);
%
%validate response values: integer and in range, otherwise can't proceed
%
resps=data_comp(:,ncols-1);
trials=data_comp(:,ncols);
if any(resps~=round(resps)) | any(trials~=round(trials)) | any(resps<0) | any(resps>trials) | any(trials<0)
    wmsg=sprintf('response counts or trial counts is non-integer or out of range');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
%
%validate indices for choices: integer and in range, otherwise can't proceed
%
all_stim_indices=unique(data_comp(:,1:ncols-2));
if any(all_stim_indices~=round(all_stim_indices)) | any(all_stim_indices<1) | any(all_stim_indices>length(aux_out.typenames))
    wmsg=sprintf('stimulus indices are non-integer or out of range, should be in [1:%1.0f]',length(aux_out.typenames));
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
%
% stimuli that never appear as indices: warning but can proceed
%
missing_stims=setdiff(1:length(aux_out.typenames),all_stim_indices);
if ~isempty(missing_stims) 
    wmsg=sprintf('some stimulus indices never appear in choices');
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
    disp('missing indices:')
    disp(sprintf(' %1.0f',missing_stims))
end   
%
%validate choice type: if not recognized, cannot proceed
%
if_valid_choice_type=strmatch(aux_out.opts_read.choice_type,valid_choice_types,'exact');
if isempty(if_valid_choice_type)
    wmsg=sprintf('choice type (%s) not recognized',aux_out.opts_read.choice_type);
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
%
if aux_out.warn_bad>0
    disp('cannot proceed');
    disp(aux_out.warnings);
    return
end
%
%consolidate if requested
%
if aux.opts_read.if_consolidate
    data_comp_orig=data_comp;
    switch aux_out.opts_read.choice_type
        case 'triadic'
            flip=find(data_comp(:,2)>data_comp(:,3));
            data_comp(flip,[2 3])=data_comp(flip,[3 2]); %change the indices
        case 'tetradic'
            %first put comparisons in lexicographic order
            rev12=find(data_comp(:,1)>data_comp(:,2));
            data_comp(rev12,[1 2])=data_comp(rev12,[2 1]);
            rev34=find(data_comp(:,3)>data_comp(:,4));
            data_comp(rev34,[3 4])=data_comp(rev34,[4 3]);
            %
            flip_first=find(data_comp(:,1)>data_comp(:,3));
            flip_tie=intersect(find(data_comp(:,1)==data_comp(:,3)),find(data_comp(:,2)>data_comp(:,4)));
            flip=union(flip_first,flip_tie);
            data_comp(flip,[1 2 3 4])=data_comp(flip,[3 4 1 2]); %change the indices
    end
    nflipped=length(flip);
    data_comp(flip,ncols-1)=data_comp(flip,ncols)-data_comp(flip,ncols-1);
    [urows,ia,ic]=unique(data_comp(:,1:ncols-2),'rows','stable');
    data_comp_dups=data_comp;
    data_comp=zeros(length(ia),ncols);
    for k=1:length(ia)
        rows=find(ic==k);
        data_comp(k,1:ncols-2)=data_comp_dups(rows(1),1:ncols-2);
        data_comp(k,ncols-1:ncols)=sum(data_comp_dups(rows,ncols-1:ncols),1);
    end
    if aux.opts_read.if_log
        disp(sprintf('consolidation: %4.0f sets of judgments consolidated to %4.0f; %4.0f sets flipped',size(data_comp_orig,1),size(data_comp,1),nflipped));
    end      
end
return
end

`rs_read_coorddata(fullname, aux)`

[data_out,aux_out]=rs_read_coorddata(fullname,aux) reads a coordinate file into a dataset structure with a single record

This is the preferred method for bringing coordinates from a single file into a dataset structure suitable for display and geometrical analysis.

For files generated by analysis of similarity judgments via [??how to refer to Python package], this will also import the goodness-of-fit data.
For multiple coordinate sets collected with the same experimental paradigm, use rs_get_coordsets, or use this routine and combine multiple dataset structures into a single dataset structure with rs_concat_coordsets.
A ray structure will be created.
To create a coordinate structure based on a quadratic form model, also use rs_get_coordsets.
For binary texture domain data with symmetry augmentation, use rs_get_coordsets.
To import coordinates generated externally or that exist as arrays rather than in a file, use rs_import_coordsets.

Input arguments:

fullname (char or singleton char array) –

full file name with path; if empty, it will be requested interactively. File names must contain the string '_coords'. See note below regarding setup files.
aux (struct) –
a structure, can be omitted, with fields
- opts_read (struct) – options for reading, can be omitted, with fields
  - if_gui (int) – 1 to use graphical interface to request data file name if 'fullname' is empty, 0 to use console; default is 1; see note below regarding customization
  - if_log (int) – 1 to log progress, 0 to omit; default is 1; see note below regarding customization
  - data_fullname_def (char) – prompt for data file if fullname' is empty; see note below regarding customization
  - ui_filter (char) – file name filter that appears in graphical user interface when if_gui=1; defaults to '_coords';see note below regarding customization
- opts_rays (struct) – options for identifying rays, may be omitted; see rs_findrays for details
- opts_check (struct) – options for consistency checking, with field
  - if_warn (int) – 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1

Output arguments:

data_out ( struct ) –
dataset structure with one record, and fields
- ds (singleton cell array) – coordinate structure, ds{1}{idim} is an array of [nstims idim] of coordinates for the sole record
- sas (singleton cell array) – stimulus metadata structure, sas{1} is the stimulus metadata for the sole record
- sets (singleton cell array) – set metadata structure, sets{1} is the response metadata for the sole record
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- rayss (cell array of struct) – rayss{1} is the ray structure for the sole record
- opts_read (cell array of struct) – opts_read{1} is aux.opts_read, with defaults filled in
- opts_rays (cell array of struct) – opts_rays{1} are the options used for creating the ray structure for the sole record
- opts_qpred (cell array of struct) – empty (included for compatibility with rs_get_coordsets)
- syms_list (struct) – empty (included for compatibility with rs_get_coordsets)
- opts_check (struct) – aux.opts_check, with defaults filled in

Note regarding domain names, stimulus coordinates, and setup files

The domain name is determined from the data file name based on the string preceding '_coords' (or, opts_read.coord_string, if customized). This in turn determines how stimulus coordinates are specified.
- For non-reserved domain names (opts_read.type_class_aux, or any domain in opts_read.domain_list_def), stimulus coordinates are specified by 'type_coords' and no setup file is used.
- Other domain names are reserved, and stimulus coordinates are specified by a 'setup file', whose suggested name is constructed from the data file name up to the string preceding opts_read.coord_string, and appending opts_read.setup_suffix
- The domain name is also used to determine a 'type_class', which is used internally. This is carried out in psg_coorddata_parsename, as follows:
  - The domains 'faces*' are reserved; type_class is faces_mpi; used with a setup file
  - The domain 'irgb' is reserved; type_class is 'irgb'; used with a setup file
  - The domain 'mater' is reserved; type_class is 'mater'; used with a setup file
  - The domain opts_read.type_class_aux is NOT reserved; type class is type_class_aux; no setup file used
  - Any domain that is listed in opts_read.domain_list_def is NOT reserved; type class is 'domain'; no setup file used
- Otherwise, type_class is set to opts_read.type_class_def, ('btc' unless customized); used with a setup file
When a dataset structure is written by rs_write_coordsets, the setup file information, if any, will be embedded. When that file is read in and an embedded setup file is detected, no setup file is requested.
Also see note below regarding customization

Note regarding customization

The defaults for the following parameters can be set by editing the line containing generic.opts_read.[param_name] in rs_aux_defaults_define, running it once, and saving the workspace as rs_aux_defaults.mat.
- if_gui
- ui_filter
- if_log
- data_fullname_def
- setup_fullname_def
- domain_list_def
- setup_suffix
- permutes_ok
- coord_string
- type_class_aux

See also: rs_import_coordsets, rs_aux_customize, rs_findrays, rs_check_coordsets, PSG_READ_COORDDATA, PSG_MAKE_SETSTRUCT, PSG_FINDRAYS_SETOPTS, PSG_FINDRAYS.

Source code in src/rs_read_coorddata.m

function [data_out,aux_out]=rs_read_coorddata(fullname,aux)
% [data_out,aux_out]=rs_read_coorddata(fullname,aux) reads a `coordinate file` into a `dataset structure` with a single record
%
% This is the preferred method for bringing coordinates from a single file into a `dataset structure` suitable for display and geometrical analysis.
%
%  - For files generated by analysis of similarity judgments via [??how to refer to Python package], this will also import the goodness-of-fit data.
%  - For multiple coordinate sets collected with the same experimental paradigm, use `rs_get_coordsets`, or use this routine and combine multiple `dataset structures` 
%    into a single `dataset structure` with `rs_concat_coordsets`.
%  - A `ray structure` will be created.
%  - To create a `coordinate structure` based on a `quadratic form model`, also use `rs_get_coordsets`.
%  - For `binary texture domain` data with symmetry augmentation, use `rs_get_coordsets`.
%  - To import coordinates generated externally or that exist as arrays rather than in a file, use `rs_import_coordsets`.
%
% Args:
%   fullname (char or singleton char array): full file name with path; if empty, it will be requested interactively. File names must contain the string '_coords'. See note below regarding setup files.
%
%   aux (struct): a structure, can be omitted, with fields 
%
%     - opts_read (struct): options for reading, can be omitted, with fields
%
%         - if_gui (int): 1 to use graphical interface to request data file name if 'fullname' is empty, 0 to use console; default is 1; see note below regarding customization
%         - if_log (int): 1 to log progress, 0 to omit; default is 1; see note below regarding customization
%         - data_fullname_def (char): prompt for data file if fullname' is empty; see note below regarding customization
%         - ui_filter (char): file name filter that appears in graphical user interface when if_gui=1; defaults to '*_coords*';see note below regarding customization
%
%         - **Options related to stimulus coordinates, domains, and setup files (see notes below)** 
%         - type_coords (float 2-D array): stimulus coordinates to use for non-reserved domain names, one row for each stimulus; default is empty
%         - type_coords_def (char): stimulus coordinates to use if type_coords is unspecified or empty; options are 'none', 'zeros', 'ones', or 'eye', see `rs_import_coordsets`; default is 'none'; see `rs_import_coordsets` for further details and how to customize
%         - domain_list_def (cell array): paradigm names for generic domain experiments; default is {'cars','tools','dwellings'}; determined by customization during install
%         - paradigm_type_def (char): default paradigm type; default is determined by customization during install
%         - coord_string (char): a string in a data file name that delineates the end of the domain name; defaults to '_coords'
%         - setup_fullname_def (char): prompt for setup file name
%         - setup_suffix (char): suffix added to paradigm name to create the setup file name; default is 'S'
%         - type_class_aux (char): a specified domain name and type_class
%
%         - **Options for internal use and maintenance**
%         - permutes (struct): typically omitted; each field is a suggested ray permutation for the corresponding paradigm name, e.g., permutes.bgca=[2 1 3 4] specifies a reordering of the rays for paradigm 'bgca'
%         - permutes_ok (int): typically omitted; 1 to accept suggested ray permutation, 0 to keep standard order; default is 1; see note below regarding customization
%         - if_uselocal (int): typically omitted; 0 to use options in rs_aux_defaults, 1 is reserved for maintenance; default is 0
%
%     - opts_rays (struct): options for identifying rays, may be omitted; see `rs_findrays` for details
%
%     - opts_check (struct): options for consistency checking, with field
%
%         - if_warn (int): 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
%
% Returns:
%   data_out (struct): `dataset structure` with one record, and fields
%
%     - ds (singleton cell array): `coordinate structure`, ds{1}{idim} is an array of [nstims idim] of coordinates for the sole record
%     - sas (singleton cell array): `stimulus metadata structure`, sas{1} is the stimulus metadata for the sole record
%     - sets (singleton cell array): `set metadata structure`, sets{1} is the response metadata for the sole record
%
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - rayss (cell array of struct): rayss{1} is the `ray structure` for the sole record
%     - opts_read (cell array of struct): opts_read{1} is aux.opts_read, with defaults filled in
%     - opts_rays (cell array of struct): opts_rays{1} are the options used for creating the `ray structure` for the sole record
%     - opts_qpred (cell array of struct): empty (included for compatibility with `rs_get_coordsets`)
%     - syms_list (struct): empty (included for compatibility with `rs_get_coordsets`)
%     - opts_check (struct): aux.opts_check, with defaults filled in
%
% Note: Note regarding domain names, stimulus coordinates, and setup files
%     - The domain name is determined from the data file name based on the string preceding '_coords' (or, opts_read.coord_string, if customized).  This in turn determines how stimulus coordinates are specified.
%
%         - For non-reserved domain names (opts_read.type_class_aux, or any domain in opts_read.domain_list_def), stimulus coordinates are specified by 'type_coords' and no setup file is used.
%         - Other domain names are reserved, and stimulus coordinates are specified by a 'setup file', whose suggested name is constructed from the data file name up to the string preceding opts_read.coord_string, and appending opts_read.setup_suffix
%         - The domain name is also used to determine a 'type_class', which is used internally. This is carried out in psg_coorddata_parsename, as follows:
%
%             - The domains 'faces*' are reserved; type_class is faces_mpi; used with a setup file
%             - The domain 'irgb' is reserved; type_class is 'irgb'; used with a setup file
%             - The domain 'mater' is reserved; type_class is 'mater'; used with a setup file
%             - The domain opts_read.type_class_aux is NOT reserved; type class is type_class_aux; no setup file used
%             - Any domain that is listed in opts_read.domain_list_def is NOT reserved; type class is 'domain'; no setup file used
%
%         - Otherwise, type_class is set to opts_read.type_class_def, ('btc' unless customized); used with a setup file
%
%     - When a `dataset structure` is written by `rs_write_coordsets`, the setup file information, if any, will be embedded. When that file is read in and an embedded setup file is detected, no setup file is requested.
%     - Also see note below regarding customization
%
% Note: Note regarding customization
%     - The defaults for the following parameters can be set by editing the line containing generic.opts_read.[param_name] in  `rs_aux_defaults_define`, running it once, and saving the workspace as rs_aux_defaults.mat.
%
%         - if_gui
%         - ui_filter
%         - if_log
%         - data_fullname_def
%         - setup_fullname_def
%         - domain_list_def
%         - setup_suffix
%         - permutes_ok
%         - coord_string
%         - type_class_aux
%
%  See also: RS_IMPORT_COORDSETS, RS_AUX_CUSTOMIZE, RS_FINDRAYS, RS_CHECK_COORDSETS,
%   PSG_READ_COORDDATA, PSG_MAKE_SETSTRUCT, PSG_FINDRAYS_SETOPTS, PSG_FINDRAYS.
%
if (nargin<=1)
    aux=struct;
end
aux=filldefault(aux,'opts_read',struct);
aux.opts_read=filldefault(aux.opts_read,'type_coords',[]);
type_coords_def=getfield(getfield(rs_aux_customize(setfield(struct(),'opts_import',struct()),'rs_import_coordsets'),'opts_import'),'type_coords_def');
aux.opts_read=filldefault(aux.opts_read,'type_coords_def',type_coords_def);
%
aux=filldefault(aux,'opts_rays',struct);
%
aux=filldefault(aux,'opts_check',struct);
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
aux=rs_aux_customize(aux,'rs_read_coorddata');
%
data_out=struct;
aux_out=struct;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
if iscell(fullname)
    fullname=fullname{1};
end
%
if isempty(fullname)
    if aux.opts_read.if_gui
        if_manual=0;
        if aux.opts_read.if_justsetup
            ui_prompt='Select a setup file';
            ui_filter={cat(2,'*',aux.opts_read.setup_suffix,'.mat'),'setup file'};
        else
            ui_prompt='Select a coordinate file';
            ui_filter=aux.opts_read.ui_filter;
        end
        while (if_manual==0 & isempty(fullname))
            [filename_short,pathname]=uigetfile(ui_filter,ui_prompt,'Multiselect','off');
            if  (isequal(filename_short,0) | isequal(pathname,0)) %use Matlab's suggested way to detect cancel
                if_manual=getinp('1 to return to selection from console','d',[0 1]);
            else
                fullname=cat(2,pathname,filename_short);
            end
        end
    end
end
if aux.opts_read.if_justsetup==0
    [d,sa,opts_read_used,pipeline]=psg_read_coorddata(fullname,[],aux.opts_read);
    opts_read_used.data_fullnames={opts_read_used.data_fullname}; %for compatibility with rs_get_coordsets
    opts_read_used.setup_fullnames={opts_read_used.setup_fullname}; %for compatibility with rs_get_coordsets
    opts_read_used.input_type_desc=aux.opts_read.input_types{1}; %rs_read_coorddata only reads experimental data, which is type 1
    opts_makeset.paradigm_type_def=aux.opts_read.paradigm_type_def;
    opts_makeset.domain_list_def=aux.opts_read.domain_list_def;
    opts_makeset.if_uselocal=aux.opts_read.if_uselocal;
    sets=psg_make_setstruct('data',opts_read_used.dim_list,opts_read_used.data_fullname,sa.nstims,pipeline,opts_makeset);
    %
    [rays,wmsg,opts_rays_used]=rs_findrays(sa,opts_read_used.setup_fullname,aux.opts_rays);
    if ~isempty(wmsg)
        aux_out=rs_warning(wmsg,0,aux_out);
    end
else
    setup_fullname=fullname;
    [d,sa,opts_read_used,pipeline]=psg_read_coorddata(fullname,setup_fullname,aux.opts_read);
    d=struct;
    sets=struct;
end
%
data_out.ds{1}=d;
data_out.sas{1}=sa;
data_out.sets{1}=sets;
aux_out.opts_check=aux.opts_check;
aux_out.opts_read{1}=opts_read_used;
aux_out.opts_rays{1}=opts_rays_used;
aux_out.rayss{1}=rays;
%for compatibility with rs_get_coordsets;
aux_out.opts_qpred{1}=struct;
aux_out.syms_list=struct();
%
%check consistency
%
check=rs_check_coordsets(data_out,aux.opts_check);
if ~isempty(check.warnings) %since strvcat([],[])~=[]
    aux_out.warnings=strvcat(aux_out.warnings,check.warnings);
    warn_leadin=getfield(getfield(rs_aux_customize(struct()),'overall'),'warn_leadin');
    for k=1:size(aux_out.warnings,1)
        disp(cat(2,warn_leadin,aux_out.warnings(k,:)));
    end
end
aux_out.warn_bad=aux_out.warn_bad+check.warn_bad;
return

`rs_save_figs(fullname, whichfigs, opts)`

rs_save_figs(whichfigs,whichfigs,opts) writes one or more open figures to a fig file

Input arguments:

fullname (char) –

file name, with path; if no figure handle is given, then all figures are saved, and '_fig_n' is appended, where n is sequentially incremented
whichfigs (handle or array of handles or char) –

a figure handle, a list of handles, or 'all', or empty, or omitted; if empty, current figure; default is [] (empty)
opts (struct) –
options, can be omitted, with fields
- if_log (int) – 1 to log, 0 to omit; default is 0
- ndigits (int) – number of digits to use in file name; default is 0, which uses minimum number for available figures

`rs_save_mat(fullname, s, opts)`

opts_used=rs_save_mat(fullname,s,opts) writes the fields of a structure to a mat file

Input arguments:

fullname (char) –

file name, with path; .mat will appended if necessary
s (struct) –

a structure whose fields are to be saved
opts (struct) –
options, with field
- ver (char) – format for saving; default is '-v7'

Output arguments:

opts_used ( struct ) –

opts, with defaults filled in

`rs_showpipeline(pipeline, opts)`

depth_max=rs_showpipeline(pipeline,opts) is a utility to show the pipeline field of a set metadata structure

The pipeline field tracks the processing steps leading to a dataset structure. It may have subfields 'sets', indicating a single dataset that was modified to create this structure; 'sets_combined', indicating a group of datasets that were combined to create this structure; 'opts', the options used in the processing path; and 'file_list', the names of the files containing the data.

Input arguments:

pipeline (struct) –

the 'pipeline' field of a `set metadata structure
opts (struct) –
options, can be omitted, with fields
- depth_limit (int) – maximum depth to show; defaults is Inf
- breadth_limit (int) – maximum breadth (number of branches) to show if sets_combined is present; default is Inf
- verbosity (int) – 0 (brief), 1, or 2; default is 1
- fields_expand (cell array of char) – names of fields to expand; defaults to {'opts','file_list'}; other fields that could be added are 'sets','sets_combined'

Output arguments:

depth_max ( int ) –

maximum depth reached (cannot exceed opts.depth_limit)

`rs_symumi_choicedata(data_comp, aux)`

[su,aux_out]=rs_symumi_choicedata(data_comp,aux) analyzes a set of triadic choices to determine the symmetry index and the ultrametric index (measures of consistency with symmetry and the ultrametric inequality) as described in Ordinal Characterization of Similarity Judgments on arXiv and Mathematical Neuroscience and Applications

The analysis is carried out for a range of criteria for the triads to include, and for Dirichlet fits to the choice probability distribution based on all triads (in 'su.global'), or only the triads that meet threshold criteria (in 'su.private'). See note below regarding threshold criteria and global vs. private analyses.

Input arguments:

data_comp (int 2-D array) –
Triadic choice data, with each row containing the data from a single kind of comparison
- col 1 is reference, col 2 is s1, col 3 is s2
- col 4: number of times the first difference was judged more similar than the second difference
- col 5: number of times the comparison was made
aux (struct) –
a structure, can be omitted, with fields
- opts_symumi (struct) – options for analysis, with fields
  - if_log (int) – 1 to log progress, 0 to omit; default is 1
  - h_fixlist (float 1-D array) – values for discrete component 'h', should include zero and be in ascending order, default is [0 0.001 0.01 0.1]
  - ntriplets_min (int) – minimum number of triplets for an analysis, default is 3
  - if_private (int) – 1 to also do calculations with Dirichlet fits only to the triads that meet threshold criteria, 0 to omit; default is 0
- opts_check (struct) – options for consistency checking, with field
  - if_warn (int) – 1 to show warnings, 0 to suppress; default is 1

Output arguments:

su ( struct ) –
analysis results, a structure with fields
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_symumi (struct) – aux,opts_symumi with defaults and values used
- opts_check (struct) – aux.opts_check, with defaults filled in
- opts_dirfit_a (struct) – options used for rs_dirfit_choicedata for fitting Dirichlet parameter 'a'
- opts_dirfit_ah (struct) – options used for rs_dirfit_choicedata for fitting Dirichlet parameters 'a' and 'h'
- opts_triplike (struct) – options used for psg_umi_triplike
- fig_handles (cell 1-D array) – handles to figures for summary plot ((present only if if_plot>=1)
- fig_handle_detailed (handle) – handle to figure for detailed plot (present only if if_plot=2)

Note regarding thresholds and global vs. private analyses

Triplets are screened by a threshold criterion based on the number of trials before inclusion in the calculation of the symmetry and ultrametric indices.
The criterion is applied three ways: to the minimum numnber of trials of the three triads in a triplet, the maximum number, and the average number
For the 'global' analysis, the selected triplets are used to calculate the symmetry and ultrametric indices, but all triads are used to calculate the Dirichlet parameters
For the 'private' analysis, the selected triplets are used to calculate the symmetry and ultrametric indices and also to calculate the Dirichlet parameters
The 'private' analysis is substantially slower than the 'global' analysis, and values are of the indices are typically similar; it is only enabled by setting opts_symumi.if_private=1

Triads, trials, and triplets

A triad is a set of three stimuli used in a triadic judgment: one stimulus is the reference, and the other two are the comparison stimuli
A trial is a single judgment of similarity for a given triad
A triplet is a set of three triads built out of the same three stimuli, in which each stimulus in turn serves as the reference
The number of trials in a triplet is the sum of the number of trials in its three triads

See also: rs_dirfit_choicedata, rs_addtree_choicedata, PSG_TRIPLET_CHOICES, loglik_beta_discrete.

Source code in src/rs_symumi_choicedata.m

function [su,aux_out]=rs_symumi_choicedata(data_comp,aux)
% [su,aux_out]=rs_symumi_choicedata(data_comp,aux) analyzes a set of triadic choices to determine the symmetry index and the ultrametric index
% (measures of consistency with symmetry and the ultrametric inequality) as described in Ordinal Characterization of Similarity Judgments on [arXiv](https://arxiv.org/abs/2310.07543)
% and [Mathematical Neuroscience and Applications](https://mna.episciences.org/16310/pdf)
%
% The analysis is carried out for a range of criteria for the triads to include, and for Dirichlet fits to the choice probability distribution
% based on all triads (in 'su.global'), or only the triads that meet threshold criteria (in 'su.private').  See note below regarding threshold criteria and global vs. private analyses.
%
% Args:
%   data_comp (int 2-D array): Triadic choice data, with each row containing the data from a single kind of comparison
%
%      - col 1 is reference, col 2 is s1, col 3 is s2
%      - col 4: number of times the first difference was judged __more similar than__ the second difference
%      - col 5: number of times the comparison was made
%
%   aux (struct): a structure, can be omitted, with fields 
%
%     - opts_symumi (struct): options for analysis, with fields
%
%         - if_log (int): 1 to log progress, 0 to omit; default is 1
%         - h_fixlist (float 1-D array): values for discrete component 'h', should include zero and be in ascending order, default is [0 0.001 0.01 0.1]
%         - ntriplets_min (int): minimum number of triplets for an analysis, default is 3
%         - if_private (int): 1 to also do calculations with Dirichlet fits only to the triads that meet threshold criteria, 0 to omit; default is 0
%
%         - **Plotting and replotting**
%         - if_plot (int): 1 for basic plot and statistical summary, 2 for detailed plot, 0 to omit; default is 1
%         - plot_label (char): string for plot label, default is 'ordinal analysis'
%         - su (struct): include to replot a previous analysis; otherwise omit
% 
%         - **Options for statistics and shuffles**
%         - if_frozen (int): random number control; 1 for same numbers every run, 0 for different random numbers each run, negative integer for a fixed seed each run, default is 1
%
%         - **Options to control optimization details**
%         - a_limits (float): allowed range for 'a' (shape), when fitting only 'a', default is [10^-2 10^2]
%         - a_optimset (struct): non-default optimizations parameters for fitting 'a', with `fminbnd`, default is struct()
%         - ah_optimset (struct): non-default optimizations parameters for fitting 'a' and 'h', with `fminsearch`, default is struct()
%
%         - **Options for internal use and maintenance**
%         - if_fast (int): use hard-coding for partitions, skip calls to filldefault,  default is 1
%         - if_check (int): 1 to compare methods, -1 to log, treated as 0 if if_fast=1, default is 0
%         - tol (float): tolerance for checking consistency, ignored if if_check is 0, default is 10^-5
%         - if_vec (int): if present, use vectorized method, default is abesent
%         - if_partition (struct): if present, use general calculation of psg_ineq_logic, default is absent
%
%     - opts_check (struct): options for consistency checking, with field
%
%         - if_warn (int): 1 to show warnings, 0 to suppress; default is 1
%
% Returns:
%   su (struct): analysis results, a structure with fields
%
%     - **Symmetry and ultrametric indices**
%     - summary (cell 1-D array): summary of analysis of symmetry and ultrametric indices, organized by fraction of triplets retained; summary{1} is analysis with fixed value of 'h', summary{2} is analysis with 'h' fitted; summary{:}.sym and summary{1}.umi contain the following subfields
%
%         - params (struct): params.a and params.h are the Dirichlet parameters
%         - apriori_vals (float): a priori value of the index
%         - ah_llr (float): log likelihood ratio for the Dirichlet fit to the choice probability distribution
%         - thr_type (cell 1-D array): statistics for threshold type 1 (min), 2 (max), 3 (avg)
%
%             - means_per_set_adj (float 2-D array): means_per_set_adj(ithr,[1 2 3]) is the mean index for each threshold type, adjusted by log(h) for umi index
%             - eb_stds (float 2-D array): 1 s.d. error bar size
%             - frac_keep_list (float 1-d array): fraction of triplets kept
%             - tally_table (int 2-D array): tally_table(ithr,[1 2 3]) is the threshold, count of triads, count of trials
%             - thr_ptr_use (int 1-D array): pointers into thresholds (tally_table(:,1)), corresponding to values in frac_keep_list
%
%     - global (struct): detailed likelihood analysis for symmetry and ultrametric indices, based on Dirichlet fits to choice probabilities for all triadic judgments, with fields
%
%         - a (int 3-D array): a(1,1:2,ih) is the fitted Dirichlet shape parameter 'a' and log likelihood per trial assuming h=h_fixlist(ih)
%         - ah (int 2-D array): ah(1,1:3) are the jointly fitted Dirichlet parameters 'a' and 'h' and log likelihood per trial for threshold value dirichlet.tallies(ithr,1)
%         - sym_hfixed (cell 2-D array): sym_hfixed{isv,ithr_type}(ithr,:,ih) is sum (isv=1) or the variance (isv=2) of the symmetry index, for threshold type ithr_type, threshold value dirichlet.tallies(ithr,1), assuming h=h_fixlist(ih)
%         - sym (cell 2-D array): sym{isv,ithr_type}(ithr,:) is the sum (isv=1) or the variance (isv=2) of the symmetry index, for threshold type ithr_type, threshold value dirichlet.tallies(ithr,1), with 'a' and 'h' jointly fitted
%         - umi_hfixed (cell 2-D array): umi_hfixed{isv,ithr_type}(ithr,:,ih) is the sum (isv=1) or the variance (isv=2) of the ultrametric index, for threshold type ithr_type, threshold value dirichlet.tallies(ithr,1), assuming h=h_fixlist(ih)
%         - umi (cell 2-D array): umi{isv,ithr_type}(ithr,:) is the sum (isv=1) or the variance (isv=2) of the ultrametric index, for threshold type ithr_type, threshold value dirichlet.tallies(ithr,1), with 'a' and 'h' jointly fitted
% 
%     - private (struct): detailed likelihood analysis for symmetry and ultrametric indices, based on Dirichlet fits only to choice probabilities that meet the threshold criterion, with fields
%
%         - a (cell 1-D array): a{ithr_type}(ithr,1:2,ih) are the fitted Dirichlet shape parameter 'a' and log likelihood per trial for threshold type and threshold value dirichlet.tallies(ithr,1), assuming h=h_fixlist(ih)
%         - ah (cell 1-D array): ah{ithr_type}(ithr,1:3,ih) are the jointly fitted Dirichlet parameters 'a' and 'h' and log likelihood per trial for threshold type ithr_type and threshold value dirichlet.tallies(ithr,1)
%         - sym_hfixed (cell 2-D array): sym_hfixed{isv,ithr_type}(ithr,:,ih) is the sum (isv=1) or the variance (isv=2) of the symmetry index, for threshold type ithr_type, threshold value dirichlet.tallies(ithr,1), assuming h=h_fixlist(ih)
%         - sym (cell 2-D array): sym{isv,ithr_type}(ithr,:) is the sum (isv=1) or the variance (isv=2) of the symmetry index, for threshold type ithr_type, threshold value dirichlet.tallies(ithr,1), with 'a' and 'h' jointly fitted
%         - umi_hfixed (cell 2-D array): umi_hfixed{isv,ithr_type}(ithr,:,ih) is the sum (isv=1) or the variance (isv=2) of the ultrametric index, for threshold type ithr_type, threshold value dirichlet.tallies(ithr,1), assuming h=h_fixlist(ih)
%         - umi (cell 2-D array): umi{isv,ithr_type}(ithr,:) is the sum sum (isv=1) or the variance (isv=2) of the ultrametric index, for threshold type ithr_type, threshold value dirichlet.tallies(ithr,1), with 'a' and 'h' jointly fitted
% 
%     - **Dirichlet fits**
%     - dirichlet (struct): Dirichlet fits for lowest and highest thresholds for number of trials in a triad, with fields
% 
%         - tallies (int 2-D array): tallies(:,1) is threshold number of trials in a triad; tallies(:,2) is number of triads meeting the threshold; tallies(:,3) is number of trials in those triads
%         - columns_tallies (cell 1-D array): labels for columns of tallies
%         - h_fixlist (float 1-D array): list of values assumed for the discrete component, first element is 0
%         - a (int 3-D array): a(ithr,1,ih) is fitted value of Dirichlet shape parameter 'a' for triads meeting threshold of tallies(ithr,1) and assuming h=h_fixlist(ih); a(ithr,2,ih) is corresponding log likelihood per trial
%         - columns_a (cell 1-D array): labels for columns of a
%         - ah (int 2-D array): ah(ithr,1:2) are jointly fitted values of Dirichlet shape parameter 'a' and discrete parameter 'h' for triads meeting threshold of tallies(ithr,1); ah(ithr,3) is corresponding log likelihoood per trial
%         - columns_ah (cell 1-D array): labels for columns of ah
% 
%     - meta (struct): labels for dimensions of the variables in su.global and su.private
%
%     - counts (struct): summary of the input data, a structure with fields
%
%         - ntrials_found (int): number of individual judgments
%         - ntriads_found (int): number of distinct triads judged
%         - nstims_found (int): number of different stimuli
%         - unique_stims (int 1-D array): list of unique stimuli
%
%     - tallies (cell 1-D array): summary of the data used for each calculation in global and private, where tallies{ithr_type}(ithr,:) is [threshold pointer, number of triplets used, number of trials used] for threshold type ithr_type (1: min, 2: max, 3: avg, see note below regarding thresholds); the threshold value corresponding to ithr is dirichlet.tallies(ithr,1)
%
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_symumi (struct): aux,opts_symumi with defaults and values used
%     - opts_check (struct): aux.opts_check, with defaults filled in
%     - opts_dirfit_a (struct): options used for `rs_dirfit_choicedata` for fitting Dirichlet parameter 'a'
%     - opts_dirfit_ah (struct): options used for `rs_dirfit_choicedata` for fitting Dirichlet parameters 'a' and 'h'
%     - opts_triplike (struct): options used for `psg_umi_triplike`
%     - fig_handles (cell 1-D array): handles to figures for summary plot ((present only if if_plot>=1)
%     - fig_handle_detailed (handle): handle to figure for detailed plot (present only if if_plot=2)
%
% Note: Note regarding thresholds and global vs. private analyses
%     - Triplets are screened by a threshold criterion based on the number of trials before inclusion in the calculation of the symmetry and ultrametric indices.
%     - The criterion is applied three ways:  to the minimum numnber of trials of the three triads in a triplet, the maximum number, and the average number
%     - For the 'global' analysis, the selected triplets are used to calculate the symmetry and ultrametric indices, but all triads are used to calculate the Dirichlet parameters
%     - For the 'private' analysis, the selected triplets are used to calculate the symmetry and ultrametric indices and also to calculate the Dirichlet parameters 
%     - The 'private' analysis is substantially slower than the 'global' analysis, and values are of the indices are typically similar; it is only enabled by setting opts_symumi.if_private=1
% 
% Note: Triads, trials, and triplets
%     - A triad is a set of three stimuli used in a triadic judgment: one stimulus is the reference, and the other two are the comparison stimuli
%     - A trial is a single judgment of similarity for a given triad
%     - A triplet is a set of three triads built out of the same three stimuli, in which each stimulus in turn serves as the reference
%     - The number of trials in a triplet is the sum of the number of trials in its three triads
% 
% See also: RS_DIRFIT_CHOICEDATA, RS_ADDTREE_CHOICEDATA, PSG_TRIPLET_CHOICES, LOGLIK_BETA_DISCRETE.
%
if (nargin<=1)
    aux=struct;
end
aux=filldefault(aux,'opts_symumi',struct);
aux.opts_symumi=filldefault(aux.opts_symumi,'if_log',1);
aux.opts_symumi=filldefault(aux.opts_symumi,'h_fixlist',[0 0.001 0.01 0.1]);
aux.opts_symumi=filldefault(aux.opts_symumi,'ntriplets_min',3);
aux.opts_symumi=filldefault(aux.opts_symumi,'if_private',0);
%
aux.opts_symumi=filldefault(aux.opts_symumi,'if_frozen',1);
aux.opts_symumi=filldefault(aux.opts_symumi,'a_limits',[10^-2 10^2]);
aux.opts_symumi=filldefault(aux.opts_symumi,'a_optimset',struct());
aux.opts_symumi=filldefault(aux.opts_symumi,'ah_optimset',struct());
%
aux.opts_symumi=filldefault(aux.opts_symumi,'if_fast',1);
aux.opts_symumi=filldefault(aux.opts_symumi,'if_check',0);
aux.opts_symumi=filldefault(aux.opts_symumi,'if_tol',10^-5);
%
aux.opts_symumi=filldefault(aux.opts_symumi,'if_plot',1);
aux.opts_symumi=filldefault(aux.opts_symumi,'plot_label','ordinal analysis');
%
aux=filldefault(aux,'opts_check',struct);
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
aux=rs_aux_customize(aux,'rs_symumi_choicedata');
%
aux_out=struct;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
if isfield(aux.opts_symumi,'su') 
    if aux.opts_symumi.if_plot>0
        [aux_out.fig_handles,aux_out.fig_handle_detailed,aux_out.summary]=rs_symumi_plot(data_comp,aux.opts_symumi.su,aux);
    end
    su=aux.opts_symumi.su;
    return
end
dirfit_opts={'a_limits','a_optimset','ah_optimset','if_frozen','if_log'}; %options to transfer from aux.opts_symumi to aux_dirfit.opts_dirfit
triplike_opts={'if_fast','if_check','tol','if_vec','if_partition'}; %options to transfer from aux.opts_symumi to opts_triplike;
%
%set up random number generator
%
if_frozen=aux.opts_symumi.if_frozen;
if (if_frozen~=0) 
    rng('default');
    if (if_frozen<0)
        rand(1,abs(if_frozen));
    end
else
    rng('shuffle');
end
%
su=struct;
%
% triadic?
%
if size(data_comp,2)~=5
    wmsg=sprintf('choice data must be triadic');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
%
choices=data_comp(:,end-1:end);
choices_nz=find(choices(:,2)>0);
choices_used=choices(choices_nz,:);
probs=choices_used(:,1)./choices_used(:,2);
choices_used=choices(choices_nz,:);
%
% enough data? at least two choices needed to fit a, three choices for a and h
%
nchoices=length(choices_nz);
choices_needed=3; %minimal choices needed
if nchoices<choices_needed
    wmsg=sprintf('insufficient choices available for fitting; at least %2.0f needed',choices_needed);
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
if all(or(probs==0,probs==1))
   wmsg=sprintf('all probabilities are 0 or 1; fitted parameter values are unreliable');
   aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
%
aux_out.opts_check=aux.opts_check;
aux_out.opts_symumi=aux.opts_symumi;
%
if aux_out.warn_bad>0
    disp('cannot proceed');
    disp(aux_out.warnings);
    return
end
%
%report number of stimulus types
%
data_nz=data_comp(choices_nz,:); %ignore choices with no trials
col_closer=4;
col_trials=5;
ntrials_found=sum(data_nz(:,col_trials));
ntriads_found=size(data_nz,1);
ustims_found=unique(reshape(data_nz(:,[1:3]),[3*ntriads_found,1]));
nstims_found=length(ustims_found);
%
su.counts=struct;
su.counts.ntrials_found=ntrials_found;
su.counts.ntriads_found=ntriads_found;
su.counts.nstims_found=nstims_found;
su.counts.unique_stims_found=ustims_found;
%
nstims=nstims_found;
%
%make stimulus indices be consecutive from 1 to nstims
data=data_nz;
indices=data_nz(:,[1:3]);
[ui,ai,ci]=unique(indices(:));
data(:,[1:3])=reshape(ci,ntriads_found,3);
%
if aux.opts_symumi.if_log
    disp(sprintf('number of unique stimuli found: %3.0f; range from %3.0f to %3.0f',nstims_found,min(ustims_found),max(ustims_found)));
    disp(sprintf('number of trials found: %6.0f',ntrials_found));
    disp(sprintf('number of triads found: %6.0f',ntriads_found));
end
%
nt=3; %number of points in a triangle
triplets=nchoosek([1:nstims],nt); %triplets: unordered subsets of 3
ntriplets=nchoosek(nstims,nt); %ntriplets: number of unordered subsets of 3
%
% ncloser: [ntriplets,3]: N(d(a,b)<d(a,c)), N(d(b,c)<d(b,a)), N(d(c,a)<d(c,b))
% ntrials: [ntriplets,3]: total trials in above
[ncloser,ntrials]=psg_triplet_choices(nstims,data); %extract triplets and sort
%
if aux.opts_symumi.if_log
    disp(sprintf('number of trials   after sorting: %6.0f',sum(ntrials(:)))) 
    disp(sprintf('number of triads   after sorting: %6.0f',sum(ntrials(:)>0)));
    disp(sprintf('number of triplets after sorting: %6.0f',sum(sum(ntrials,2)>0)));
    disp(sprintf('number of trials per triad range from %6.0f to %6.0f',min(ntrials(:)),max(ntrials(:))));
end
if (sum(ntrials(:))~=ntrials_found)
    wmsg=sprintf('mismatch of number of trials in data before sorting (%5.0f) vs after sorting (%5.0f)',ntrials_found,sum(ntrials(:)));
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
end
%
h_fixlist=aux.opts_symumi.h_fixlist;
h_fixlist=unique([0 h_fixlist(:)']);
nhfix=length(h_fixlist);
%
su.dirichlet=struct();
su.dirichlet.columns_tallies={'min_trials_per_triad','ntriads','ntrials'};
su.dirichlet.columns_a={'a','loglike_per_trial'};
su.dirichlet.columns_ah={'a','h','loglike_per_trial'};
su.dirichlet.h_fixlist=h_fixlist;
%
%Dirichlet fits, for fixed values of h and also h fitted
%code modified from psg_umi_triplike_demo, adapted for if_fixa=0, and rs_dirfit_choicedata
%
%Note that here and in private fits, values of a may differ slightly e.g., 0.01) from those obtained by psg_umi_triplike_demo
%This is because here, optimization uses loglik_beta_discrete, which adds a quadratic cost when h<0,
%while in psg_umi_triplike_demo, loptimization uses oglik_beta, which does not add a cost, but values of h<0 are replaced by the best fit with h>=0
%
aux_dirfit=struct;
for k=1:length(dirfit_opts)
    fn=dirfit_opts{k};
    if isfield(aux.opts_symumi,fn)
        aux_dirfit.opts_dirfit.(fn)=aux.opts_symumi.(fn);
    end
end
%
aux_dirfit.opts_dirfit.if_fit_a=0;
aux_dirfit.opts_dirfit.if_fit_h=0;
aux_dirfit.opts_dirfit.if_fit_ah=0;
aux_dirfit.opts_dirfit.fixed_h=0;
aux_dirfit.opts_dirfit.if_stats=0;
%
aux_dirfit.opts_check=aux.opts_check;
%
aux_dirfit_a=aux_dirfit;
aux_dirfit_a.opts_dirfit.if_fit_a=1;
%
aux_dirfit_ah=aux_dirfit;
aux_dirfit_ah.opts_dirfit.if_fit_ah=1;
%
aux_out.opts_dirfit_a=aux_dirfit_a.opts_dirfit;
aux_out.opts_dirfit_ah=aux_dirfit_ah.opts_dirfit;
%
% fit Dirichlet parameters to triplets
%
ithr=0;
for thr=[min(ntrials(:)) max(ntrials(:))] %just compute extremes, and we only need min(ntrials(:)) for the index calculation below
    triads_use=find((ntrials(:)>=thr));
    ntriads_use=length(triads_use);
    ntrials_use=sum(ntrials(triads_use));
    if (ntriads_use>=aux.opts_symumi.ntriplets_min)
        ithr=ithr+1;
        su.dirichlet.tallies(ithr,:)=[thr,ntriads_use,ntrials_use];
        data_use=[ncloser(triads_use) ntrials(triads_use)];
        if aux.opts_symumi.if_log
            disp(sprintf('fitting Dirichlet params after thresholding triads by %3.0f trials',thr))
        end
        %fixed  values of h
        for ihfix=1:nhfix
            %
            aux_dirfit_a.opts_dirfit.fixed_h=h_fixlist(ihfix);
            [dirfit_a,aux_dirfit_out_a]=rs_dirfit_choicedata(data_use,aux_dirfit_a);
            su.dirichlet.a(ithr,:,ihfix)=[dirfit_a.a.val,dirfit_a.a.llnat_per_trial];
        end
        %fit a and h
        [dirfit_ah,aux_dirfit_out_ah]=rs_dirfit_choicedata(data_use,aux_dirfit_ah);
        su.dirichlet.ah(ithr,:)=[dirfit_ah.ah.val',dirfit_ah.ah.llnat_per_trial];
    end
end
%
%analyze for ultrametric inequality and symmetry, via strategy of psg_umi_triplike_demo with conform=0 and if_fast=1
%
ipg_strings={'private','global'};
npg=length(ipg_strings);
thr_types={'min','max','avg'};
nthr_types=length(thr_types);
%
%global analyses: Dirichlet fits not adjusted based on which triads are used
%private analyses: Dirichlet fits are private to the triads used
%
su.meta=struct;
su.global=struct;
su.meta.thr_types=thr_types;
su.meta.columns_tallies={'thr','ntriplets','ntrials'};
su.meta.columns_a={'a','loglike_per_trial'}; %values of a and h determined from the selected trials
su.meta.columns_ah={'a','h','loglike_per_trial'}; %values of a and h determined from the selected trials
su.meta.columns_sym={'llr_sym_vs_sym+notsym'}; %from likrat.sym of psg_umi_triplike
su.meta.columns_umi={'llr_umi_trans_vs_trans'}; %from likrat.umi_trans of psg_umi_triplike
su.meta.columns_sym_hfixed=su.meta.columns_sym; %from likrat.sym of psg_umi_triplike
su.meta.columns_umi_hfixed=su.meta.columns_umi; %from likrat.umi_trans of psg_umi_triplike
su.meta.thr_types=thr_types;
su.meta.ipg_strings=ipg_strings;
su.meta.surr_types={'orig data','flip_all','flip_any'};
%
su.global.a=su.dirichlet.a(1,:,:); % values with h fixed
%compute using global a and h from unthresholded Dirichlet and save in r.su.global.ah
if su.dirichlet.ah(1,2)>=0 %use full fit if h>=0
    su.global.ah=su.dirichlet.ah(1,:);
else %otherwise use best fit with h=0
    su.global.ah=[su.dirichlet.a(1,1,1),0,su.dirichlet.a(1,2,1)];
end
if aux.opts_symumi.if_private % compute these later using private a and h, to go in r.su.private.[a|ah]{ithr_type}
    su.private=struct;
    su.private.a=cell(1,nthr_types); 
    su.private.ah=cell(1,nthr_types);
end
%
su.meta.global_private_d1={'mean of sum','variance of sum'};
su.meta.global_private_d2={'threshold type'};
ipg_min=2-aux.opts_symumi.if_private;
for ipg=ipg_min:npg
    su.(ipg_strings{ipg}).sym=cell(2,nthr_types);
    su.(ipg_strings{ipg}).umi=cell(2,nthr_types);
    su.(ipg_strings{ipg}).sym_hfixed=cell(2,nthr_types);
    su.(ipg_strings{ipg}).umi_hfixed=cell(2,nthr_types);
end
%
ncomps=3;
flipconfigs=int2nary([0:2^ncomps-1]',2);  %rows are [0 0 0;1 0 0;0 1 0;1 1 0; 0 0 1;1 0 1;0 1 1;1 1 1];
nflips=size(flipconfigs,1); %2^8
%
su.meta.llr_d1={'threshold value'};
su.meta.llr_d2=su.meta.surr_types;
su.meta.llr_d3={'hfixed'};
su.meta.nsurr=length(su.meta.surr_types);
nsurr=length(su.meta.llr_d2); %three kinds of surrogates: native, flip all, flip any
%
llr_sym=cell(nsurr,2); %summed log likelihood ratio across trials, and summed variance of total 
llr_umi=cell(nsurr,2);
llr_sym_hfixed=cell(nsurr,2);
llr_umi_hfixed=cell(nsurr,2);
surr_list={1,[1 nflips],[1:nflips]};
%
%if_fast=1: calculate probabilities for all triplets
loglik_rat_sym_all=zeros(ntriplets,nflips);
loglik_rat_umi_all=zeros(ntriplets,nflips);
loglik_rat_sym_hfixed_all=zeros(ntriplets,nflips,nhfix);
loglik_rat_umi_hfixed_all=zeros(ntriplets,nflips,nhfix);
ah=su.global.ah;
ah_fixed=[squeeze(su.dirichlet.a(1,1,:)),h_fixlist(:)];
%
opts_triplike=struct;
for k=1:length(triplike_opts)
    fn=triplike_opts{k};
    if isfield(aux.opts_symumi,fn)
        opts_triplike.(fn)=aux.opts_symumi.(fn);
    end
end
aux_out.opts_triplike=opts_triplike;
%
for itriplet=1:ntriplets %accumulate likelihood ratios from each set of triplets   
    obs_orig(:,1)=ncloser(itriplet,:)';
    obs_orig(:,2)=ntrials(itriplet,:)';
    obs_orig_flip=obs_orig(:,2)-obs_orig(:,1); 
    %
    for iflip=1:nflips %each surrogate
        obs=obs_orig;
        whichflip=find(flipconfigs(iflip,:)==1);
        obs(whichflip,1)=obs_orig_flip(whichflip);
        params.a=ah(1);
        params.h=ah(2);
        likrat=psg_umi_triplike(params,obs,opts_triplike);
        loglik_rat_sym_all(itriplet,iflip)=log(likrat.sym);
        loglik_rat_umi_all(itriplet,iflip)=log(likrat.umi_trans);
        for ihfix=1:nhfix
            params.a=ah_fixed(ihfix,1);
            params.h=ah_fixed(ihfix,2);
            likrat=psg_umi_triplike(params,obs,opts_triplike);
            loglik_rat_sym_hfixed_all(itriplet,iflip,ihfix)=log(likrat.sym);
            loglik_rat_umi_hfixed_all(itriplet,iflip,ihfix)=log(likrat.umi_trans);
        end
    end %iflip
end %itriplet
if aux.opts_symumi.if_log
    disp(sprintf('symmetry and ultrametric global calculations done'));
end
%
for ipg=ipg_min:2 %private and global, code modified from psg_umi_triplike_demo with nconform=0, if_fast=1
    if aux.opts_symumi.if_log
        disp(sprintf('%10s calculations',ipg_strings{ipg}));
    end
    for ithr_type=1:nthr_types %three kinds of thresholds: min, max, average
        if_ok=1;
        thr=0; %threshold
        ithr=1; %threshold pointer
        if aux.opts_symumi.if_log
            disp(sprintf('analyzing symmetry and ultrametric likelihood ratio for threshold type %s',thr_types{ithr_type}));
        end
        nuse_prev=-1; %will allow for reuse if increasing the threshold doesn't change the number of triplets/tents used
        while (if_ok)
            switch thr_types{ithr_type}
                case 'min'
                    triplets_use=find(min(ntrials,[],2)>=thr);
                    thr_val=thr;
                case 'max'
                    triplets_use=find(max(ntrials,[],2)>=thr);
                    thr_val=thr;
                case 'avg'
                    triplets_use=find(sum(ntrials,2)>=thr);
                    thr_val=thr/3; %average not total
            end
            if (length(triplets_use)>=aux.opts_symumi.ntriplets_min)
                ntriplets_use=length(triplets_use);
                if ntriplets_use~=nuse_prev
                    did_or_skipped='did'; %have to calculate
                    nuse_prev=ntriplets_use;
                    ntrials_use=sum(sum(ntrials(triplets_use,:)));
                    su.tallies{ithr_type}(ithr,:)=[thr_val ntriplets_use ntrials_use]; %threshold, number of triplets, number of trials
                    %compute private best-fitting a and h
                    data_use=[reshape(ncloser(triplets_use,:),3*ntriplets_use,1) reshape(ntrials(triplets_use,:),3*ntriplets_use,1)];
                    if (ipg==1)
                        %private fits, assuming fixed values of h
                        for ihfix=1:nhfix
                            aux_dirfit_a.opts_dirfit.fixed_h=h_fixlist(ihfix);
                            [dirfit_a,aux_dirfit_out_a]=rs_dirfit_choicedata(data_use,aux_dirfit_a);
                            su.private.a{ithr_type}(ithr,:,ihfix)=[dirfit_a.a.val,dirfit_a.a.llnat_per_trial];
                        end
                        %private fit for a and h
                        [dirfit_ah,aux_dirfit_out_ah]=rs_dirfit_choicedata(data_use,aux_dirfit_ah);
                        if dirfit_ah.ah.val(2)>=0 %ensure h >=0
                            su.private.ah{ithr_type}(ithr,:)=[dirfit_ah.ah.val',dirfit_ah.ah.llnat_per_trial];
                        else
                            su.private.ah{ithr_type}(ithr,:)=[dirfit_a.a.val,0,dirfit_ah.ah.llnat_per_trial];
                        end
                    end
                    %
                    %fast global option:calculate probabilities for all triplets and later select
                    %
                    if ipg==2
                         loglik_rat_sym=loglik_rat_sym_all(triplets_use,:);
                         loglik_rat_umi=loglik_rat_umi_all(triplets_use,:);
                         loglik_rat_sym_hfixed=loglik_rat_sym_hfixed_all(triplets_use,:,:);
                         loglik_rat_umi_hfixed=loglik_rat_umi_hfixed_all(triplets_use,:,:);
                    else % ipg=1 (private)
                        ah=su.private.ah{ithr_type}(ithr,:); %a and h both fitted
                        ah_fixed=[squeeze(su.private.a{ithr_type}(ithr,1,:)),h_fixlist(:)]; %a fitted, h fixed
                        loglik_rat_sym=zeros(ntriplets_use,nflips);
                        loglik_rat_umi=zeros(ntriplets_use,nflips);
                        loglik_rat_sym_hfixed=zeros(ntriplets_use,nflips,nhfix);
                        loglik_rat_umi_hfixed=zeros(ntriplets_use,nflips,nhfix);
                        for itriplet=1:ntriplets_use %accumulate likelihood ratios from each set of triplets
                            obs_orig(:,1)=ncloser(triplets_use(itriplet),:)';
                            obs_orig(:,2)=ntrials(triplets_use(itriplet),:)';
                            obs_orig_flip=obs_orig(:,2)-obs_orig(:,1); 
                            %
                            for iflip=1:nflips %each surrogate
                                obs=obs_orig;
                                whichflip=find(flipconfigs(iflip,:)==1);
                                obs(whichflip,1)=obs_orig_flip(whichflip);
                                params.a=ah(1);
                                params.h=ah(2);
                                likrat=psg_umi_triplike(params,obs,opts_triplike);
                                loglik_rat_sym(itriplet,iflip)=log(likrat.sym);
                                loglik_rat_umi(itriplet,iflip)=log(likrat.umi_trans);
                                for ihfix=1:nhfix
                                    params.a=ah_fixed(ihfix,1);
                                    params.h=ah_fixed(ihfix,2);
                                    likrat=psg_umi_triplike(params,obs,opts_triplike);
                                    loglik_rat_sym_hfixed(itriplet,iflip,ihfix)=log(likrat.sym);
                                    loglik_rat_umi_hfixed(itriplet,iflip,ihfix)=log(likrat.umi_trans);
                                end
                            end %iflip
                        end %itriplet
                     end %ipg
%                    %do statistics
                     for isurr=1:nsurr
                         surr_sel=surr_list{isurr}; %for isurr=1, this is just the original data (1)
                         llr_sym{isurr,1}=sum(mean(loglik_rat_sym(:,surr_sel),2),1);
                         llr_umi{isurr,1}=sum(mean(loglik_rat_umi(:,surr_sel),2),1);
                         llr_sym_hfixed{isurr,1}=reshape(sum(mean(loglik_rat_sym_hfixed(:,surr_sel,:),2),1),[1 1 nhfix]);
                         llr_umi_hfixed{isurr,1}=reshape(sum(mean(loglik_rat_umi_hfixed(:,surr_sel,:),2),1),[1 1 nhfix]);
                         if (isurr>1)
                             %each triplet contributes independently to the variance
                             %variance for each triplet is normalized by N not N-1, since we have all the values
                             llr_sym{isurr,2}=sum(var(loglik_rat_sym(:,surr_sel),1,2),1);
                             llr_umi{isurr,2}=sum(var(loglik_rat_umi(:,surr_sel),1,2),1);
                             llr_sym_hfixed{isurr,2}=reshape(sum(var(loglik_rat_sym_hfixed(:,surr_sel,:),1,2),1),[1 1 nhfix]);
                             llr_umi_hfixed{isurr,2}=reshape(sum(var(loglik_rat_umi_hfixed(:,surr_sel,:),1,2),1),[1 1 nhfix]);
                         else %isurr=1: original data. Here, goal is for psg_umi_triplike_plota to compute standard error of the mean
                             %which is sqrt(var)/ntriplets_use, but
                             %psg_umi_triplike_plota will find square root and then divide by ntriplets_use
                             %so here we just compute var, normalized by N-1 since it is a sample
                             %here, surr_sel=1
                             llr_sym{isurr,2}=var(loglik_rat_sym(:,surr_sel),0,1);
                             llr_umi{isurr,2}=var(loglik_rat_umi(:,surr_sel),0,1);
                             llr_sym_hfixed{isurr,2}=reshape(var(loglik_rat_sym_hfixed(:,surr_sel,:),0,1),[1 1 nhfix]);
                             llr_umi_hfixed{isurr,2}=reshape(var(loglik_rat_umi_hfixed(:,surr_sel,:),0,1),[1 1 nhfix]);
                         end
                         %
                         for isv=1:2% mean and variance
                             su.(ipg_strings{ipg}).sym{isv,ithr_type}(ithr,isurr)=llr_sym{isurr,isv};
                             su.(ipg_strings{ipg}).umi{isv,ithr_type}(ithr,isurr)=llr_umi{isurr,isv};
                             su.(ipg_strings{ipg}).sym_hfixed{isv,ithr_type}(ithr,isurr,:)=llr_sym_hfixed{isurr,isv};
                             su.(ipg_strings{ipg}).umi_hfixed{isv,ithr_type}(ithr,isurr,:)=llr_umi_hfixed{isurr,isv};
                         end %isv
                     end %isurr
                else %change in threshold does not change which triplets are included
                     did_or_skipped='skp';
                     su.tallies{ithr_type}(ithr,:)=su.tallies{ithr_type}(ithr-1,:);
                     su.tallies{ithr_type}(ithr,1)=thr_val; %threshold is new
                     if (ipg==1)
                        su.private.a{ithr_type}(ithr,:,:)=su.private.a{ithr_type}(ithr-1,:,:);
                        su.private.ah{ithr_type}(ithr,:)=su.private.ah{ithr_type}(ithr-1,:);
                     end
                     for isurr=1:nsurr
                         for isv=1:2% mean and variance
                             su.(ipg_strings{ipg}).sym{isv,ithr_type}(ithr,isurr)=llr_sym{isurr,isv};
                             su.(ipg_strings{ipg}).umi{isv,ithr_type}(ithr,isurr)=llr_umi{isurr,isv};
                             su.(ipg_strings{ipg}).sym_hfixed{isv,ithr_type}(ithr,isurr,:)=llr_sym_hfixed{isurr,isv};
                             su.(ipg_strings{ipg}).umi_hfixed{isv,ithr_type}(ithr,isurr,:)=llr_umi_hfixed{isurr,isv};
                         end %isv
                     end %isurr
                 end %nuse_prev
                 if aux.opts_symumi.if_log
                     disp(sprintf('%s ipg %3.0f ithr_type %3.0f ithr %3.0f thr %3.0f ntriplets_use %6.0f size(loglik_rat_sym) %6.0f %4.0f size(loglik_rat_sym_hfixed) %6.0f %4.0f %4.0f',...
                         did_or_skipped,ipg,ithr_type,ithr,thr,ntriplets_use,size(loglik_rat_sym),size(loglik_rat_sym_hfixed)));
                 end
                 thr=thr+1; %threshold
                 ithr=ithr+1; %pointer
             else
                 if_ok=0;
             end
         end %if_ok
    end %thr_type
end %ipg
%
[aux_out.fig_handles,aux_out.fig_handle_detailed,su.summary]=rs_symumi_plot(data_comp,su,aux);
return
end

`rs_typenames2colors(typenames, aux)`

[rgb,symb,cvecs,aux_out]=rs_typenames2colors(typenames,aux) is a utility that assigns a display color and symbol to a set of stimulus labels. Stimulus coordinates are extracted from 'typenames', and mapped to a color and plotting symbol detailed below. The color is determined by the direction from the origin and the symbol is determined by the sign(s) of the coordinates.

Input arguments:

typenames (char or cell array of char) –

one or more stimulus labels; see note below regarding parsing of typenames into coordinates
aux (struct) –
auxiliary input, may be omited, with field
- opts_tn2c (struct) – options controlling mapping of coordinates to colors and symbols, with fields

Output arguments:

rgb ( float 1-D array ) –

the assigned rgb color triplet
symb ( char ) –

the plotting symbol
cvecs ( float 2-D array ) –

decoded coordinates, with each row corresponding to an element of typnames, and consisting of length(coord_lets) coordinate values, with NaN if coordinate is not found
aux_out ( struct ) –
auxiliary outputs and parameter values used, with field
- opts_tn2c (struct) – aux.opts_tn2c, with defaults filled in

Note regarding parsing of typenames into coordinates

The typenames argument is a string, or a a cell array of strings; each of which is parsed into coordinates.
These strings consist of a coordinate axis designator, typically [a-j] ynless modified by opts_tn2c.coord_lets, followed by a numeric quantity.
The numeric quantity consists either of string of digits (assumed to represent a positive quantity), or a string of digits preceded by p or m (to represent a positive or negative quantity), or z (followed by any numbers), indicating zero.
The string can also contain a decimal point,'+', or '-', intrepreted in the standard fashion. The sign is multiplied by the sign designated by a preceding 'p' or 'm' if present.
Examples, assuning that coords_lets has the default value of 'abcdefghij':
- 'z' indicates the origin, i.e., 0 on all coordinate axes
- 'a3b5cm1' indicates (3,5,-1) on the first three axes
- 'cm52' indicates -52 on the c (third) axis
- 'bp4 hm36' indicates +4 on the b (second) axis, -36 on the h (eighth) axis
- 'j17dm3' indicates -3 on the d (fourth) axis, 17 on the j (tenth) axis,
- 'am1.4' and 'am-1.4' indicate -1.4 on the a (first) axis but 'am-1.4' indicates +1.4 on the a axis
All of the strings in typenames are parsed in his fashion. To assign color:
- If ony one axis is present among typenames, then its color is used.
- If more than one axis is present anong the typenames, and color values are given as rgb triples, and if_color_arith=1, then colors are weighted by the magnitudes.
- Otherwise the color of the axis that has the largest magnitude coordinate will be used, with ties broken by order of axes.

Note regarding reserved paradigm types

For certain reserved paradigm types, color and symbol assignments deviate from the behavior described here and are controlled by psg_typenames2colors.
paradigm_type='btc': this refers to stimuli from the binary texture domain . Symbols are assigned as described above. 'cvecs' are decoded based on coord_lets='gbcdetuvwa', and typename strings such as 'gm0461' are interpreted as a value of 0.461 on the g (first) axis. Colors are assigned to coordinates as follows:
- g (gamma)->gray
- b,c (cardinal beta) -> dark and light blue
- d,e (diagonal beta) -> dark and light green
- t,u,v,w (theta): -> yellows and browns
- a (alpha) -> red
paradigm_type='faces': this refers to stimuli from the MPI faces domain. Strings in typenames should correspond to file names from the MPI archive, which are of the form '132_y_f_n_a' ('[individual][age][gender]_[exemplar]'). Colors and symbols are assigned as below; 'cvecs' is returned empty.
- gender: f->red [1 0 0], m->blue [0 0 1]
- age: y,m,o -> dark, medium, light (blend gender color with [.75 1 .75], weighing gender color by 0.4 (y), 0.7 (m), or 1.0(o))
- expression: neutral-> 'x' for set A, '+' for set b; angry->'*', sad -> 'v' (downward triangle), happy -> '^' (upward triangle), fright-> 'h' (hexagon), disgust -> 'p' (pentagon)
The list of default reserved paradigms can be changed by adding a line defining generic.opts_tn2c.paradigms_reserved in rs_aux_defaults_define, running it once, and saving the workspace as rs_aux_defaults.mat.

`rs_vara_coordsets(data_in, groupings, aux)`

[vara_stats,aux_out]=rs_vara_coordsets(data_in,groupings,aux) does an analysis of variance across groups of dataset structures via shuffling by trials, and provides and displays statistics

Basic strategy: records are grouped, consensus is found within each group, and the variance around the consensus is computed. Then, records are shuffled between groups. The shuffling can also be constrained by assigning tags to each dataset, e.g., according to the subject ID, and only shuffling within records that have the same tag. For each shuffle, the variance around the consensus within each group is again calculated, and compared tothat of the original data.

Note that in contrast to rs_knit_coordsets, in which shuffling permutes the coordinates of the stimuli within a record, here the shuffling permutes the records across groups.

Global consensus and consensus of groups are not returned; these can be computed with rs_knit_coordsets.

Input arguments:

data_in (struct) –
dataset structure to be processed, with fields
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record
groupings (struct) –
structure specifying assignment of records to groups, and, optionally, tags to restrict shuffling
- gps (int 1-D array) – gps(k) is the group assignment of record k, an integer from 1 to the number of groups (ngps); All groups must have at least one record.
- tags (int 1-D array) – tags(k) is the tag assigned to record k. If non-empty, shuffles will only swap a record with a record of the same tag. Defaults to empty ([]),
aux (struct) –
auxiliary inputs, may be omitted, with fields
- opts_vara (struct) – options for the variance analysis, with fields
- opts_check (struct) – options for consistency checking, with field
  - if_warn (int) – 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
- opts_align (struct) – options for alignment of data, typically omitted
- sa_pooled (struct) – include to avoid recalculation of alignment, otherwise omit; see note below regarding recalculation of alignment
- data_align (struct) – include to avoid recalculation of align ment, otherwise omit; see note below regarding recalculation of alignment

Output arguments:

vara_stats ( struct ) –

statistics and analysis parameters, detailed in note below regarding statistics and plots
aux_out ( struct ) –
auxiliary outputs and parameter values used
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_vara (struct) – aux.opts_vara, with defaults filled in
- opts_check (struct) – aux.opts_check, with defaults filled in
- opts_pcon (cell array of struct) – opts_pcon{idim} are the options used in Procrustes alignment for model dimension idim
- opts_align (struct) – aux.opts_align, with defaults filled in
- vara_stats_setup (struct) – parameters extracted from aux.opts_vara, along with the additional fields below; see note below regarding replotting a previous analysis
  - nsets (int) – number of records in data_in
  - nstims (int) – number of stimuli
  - dataset_labels (cell array of char) – dataset labels, from data_in.sets{:}.label
  - stimulus_labels (cell array of char) – stimulus labels, from data_out.sas{1}.typenames
- fig_handle (handle) – handle to figure created (present only if statistics are plotted)

General notes

For all records with data_in.sets{k}.type='data', the strings in data_in.sets{k}.paradigm_type must agree.

Note regarding statistics and plots

If aux.opts_vara.if_stats=1, statistics regarding variances between individual sets and within-group consensus, and between individual sets and the global consensus, are returned in vara_stats, along with analysis parameters
vara_stats is a structure with the following fields (fields names ending with _shuff will be omitted if nshuffs=0; rmsd=root-mean-squared deviation):
- groupings (struct): grouping information
  - ngps (int): number of groups
  - gps (int 1-D array): gps(k) is the group assignment of record k, an integer from 1 to ngps
  - gp_list (cell array): gp_list{igp} are the indices of the records in group igp
  - nsets_gp (int 1-D array): nsets_gp(igp) is the number of records in group igp
  - nsets_gp_max (int): maximum size of a group
  - tags (int 1-D array): tags(k) is the tag for record k
- nshuffs (int): number of shuffles used
- shuffs (int 2-D array): shuffles used; shuffs(ishuff,k) is the record used in place of original record k on shuffle ishuff
- opts_multi_used (struct): options used for generating shuffles, see multi_shuff_groups for details
- rmsdev_setwise (float 2-D array): rmsdev_setwise(ip,irec) is the rmsd of coordinates in record irec from the global consensus based on data_in.ds{:}{ip}, across all stimuli
- rmsdev_stmwise (float 2-D array): rmsdev_stmwise(ip,istm) is the rmsd of coordinates for stimulus istm from the global consensus based on data_in.ds{:}{ip}, across all records
- rmsdev_overall (float 2-D array): rmsdev_overall(ip,1) is the rmsd of coordinates from the global consensus based on data_in.ds{:}{ip}, across all records and stimuli
- rmsdev_setwise_gp (float 3-D array): rmsdev_setwise_gp(ip,irec,igp) is the rmsd of coordinates in group igp in record irec from its within-group consensus, based on data_in.ds{:}{ip}, across all stimuli
- rmsdev_stmwise_gp (float 3-D array): rmsdev_stmwise_gp(ip,istm,igp) is the rmsd of coordinates in group igp for stimulus istm from its within-group consensus, based on data_in.ds{:}{ip}, across all records
- rmsdev_overall_gp (float 3-D array): rmsdev_overall_gp(ip,1,igp) is the rmsd of coordinates in group igp from the within-group consensus, based on data_in.ds{:}{ip}, across all records and stimuli
- rmsdev_grpwise (float 3-D array): rmsdev_grpwise(ip,1,1) is the rmsd of coordinates from the within-group consensus, across all records and stimuli, averaged over all groups
The counts for each of these calculations are counts_[overall|setwise|stmwise] and counts_gp_[overall|setwise|stmwise], with first array coordinate fixed at 1 (since dimension does not matter)
The available rmsd (from zero) is given by rmsavail_[overall|setwise|stmwise].
If aux_out.vara_stats.nshuffs>0, then a parallel computation is done after random shuffles of the stimulus labels within each record, and the results are returned in rmsdev_[overall|setwise|stimwise]_gp_shuff and rmsdev_grpwise_shuff. For the shuffled quantities, the first three dimensions are the same as for the unshuffled quantities; and dimension 4 (length: nshuffs) is which shuffle. The latter is the key quantity: rmsdev_grpwise_shuff(:,1,1,ishuff), to be compared with the unshuffled values, rmsdev_grpwise, indicates whether the data are more clusted into groups than would be expected by chance.
To control whether the same random number seed is used on each run, use aux.opts_vara.if_frozen (default is 1).
if aux.opts_vara.if_plot=1 (default if if_stats=1), then a figure is created, with three panels:
- a heatmap of rmsdev_setwise as a function of dimension
- a heatmap of rmsdev_setwise_gp as a function of dimension
- a comparison of the rmsd from the group consensus (rmsdev_grpwise, black), overall available rmsd (blue), and quantiles (rmsdev_grpwise_shuff, red) of the rmsd from the group consensus in the shuffled data (quantiles are specified by shuff_quantiles; if nshuffs=0, shuffled values will not be plotted)

Note regarding Procrustes consensus algorithm

To find a consensus set of coordinates, the coordinates in each record of data_in are rotated, and optionally translated (if allow_offset=1), scaled (if allow_scale=1), and reflected (if allow_reflection=1). These transformations are carried out for separately for each set dimension for which coordinates are present in all of the records, i.e., for which data_in.ds{k}{idim} exists for all k.
The algorithm, in procrustes_consensus.m, is iterative. Briefly, after an initial guess is determined, a Procrustes transformation is found that minimizes the rms deviation between each record and the current guess. The guess is then revised by setting each stimulus' coordinates equal to the centroid of the coordinates of that stimulus across the records. To avoid drift of the updated guess, it is Procrustes-transformed for closest match to an alignment coordinate set (the alignment set, unless otherwise specified, is equal to the initial guess).
The iteration ends when either the number of iterations exceeds max_niters (default=1000), or the rms change of the guess is less than max_rmstol (default=10^-5)
There are several choices for initialization and alignment.
- For most purposes, the default initialization method (pcon_init_method=0) can be used, which uses the principal components of all the stimulus coordinates in all of the records. These can be optionally forced to be centered (pcon_init_method=-1) or not (pcon_init_method=-2); if unspecified (default), centering is determined by allow_offset. For these choices, if_initpca_rot=1 rotates the initial guess to match the data, or not. The default for if_initpca_rot is 1 unless any of dim_list_out>dim_list_in, in which case it is 0. The heuristic for not rotating if dim_list_out>dim_list_in, i.e., two or more sets of coordinates are to be knit together to construct a coordinate set with a greater number of dimensions, is that without rotation, the principal components reflect projections of the coordinates that are present in any of the records.
- Alternatively, pcon_init_method=r, r>0, specifies that the coordinates in data_in{r}{idim} are used.
- If pcon_init_method='specify', then pcon_initial_guess{idim} is an array of size [npts ids] for the initial guess, and pcon_alignment{idim}, which defaults to pcon_initial_guess, is used for the alignment at the end of each iteration. pcon_initial_guess and pcon_alignment may be omitted, in which case random values are used. To control whether the same random number seed is used on each run, use if_frozen (default is 1, which uses the same random numbers on each run).
- The solution is only unique up to rotation (and translation and reflection, if these components are allowed). The ambiguity is resolved by matching the consensus solution to the initial guess (or, pcon_alignment{idim} if separately supplied with pcon_init_method=0), as described above.
- Under some circumstances (e.g., several solutions that are nearly equally good), the solution found by the algorithm may depend on the initialization choice. A simple strategy to check for this is to compare the results with the default pcon_init_method=0 to the results with pcon_init_method='specify' and if_frozen=0. There are two ways that this dependency can happen.
  - One is that the number of overlapping stimuli is too small. For example, at least m points are required to determine a rotation and translation in an m-dimensional space; if there are fewer overlaps, then a consensus will still be found but there are many other consensus datasets that fit equally well.
  - A second way is that there are a sufficient number of points, but there are several solutions that are approximately equally good. Under these circumstances, the algorithm may get stuck in a local minimum. This possibility only occurs when there are at least three records in data_in, as the procedure reduces to the standard Procrustes algorithm, which finds the consensus when there are only two records, is deterministic other than does rotational ambiguity.

Note regarding replotting a previous analysis

To replot a previous calculation with additional customization or to make a composite figure, proceed as follows:
- data_in and groupings should be equal to that used in the previous calculation.
- aux.vara_stats should be equal to vara_stats from the previous calculation
- aux.vara_stats_setup should be equal to aux_out.vara_stats_setup from the previous calculation
The following modifications are allowed in fields of vara_stats_setup:
- dataset_labels (cell array of char): dataset labels; default is data_in.sets{:}.label
- stimulus_labels (cell array of char): stimulus labbels; default is data_out.sas{1}.typenames
- shuff_quantiles (float 1-d array): quantiles to show; default is 0.01 0.05 0.5 0.95 0.99
- fig_handle (handle): figure handle to plot into; figure will be created if not supplied
- row (int): row to plot into; default is 1
- nrows (int): number of rows in the figure; default is row
No further calculations are done
On return, vara_stats will be empty and aux_out.fig_handle will be the figure handle
In creating a composite figure, rows should be plotted in order from top to bottom, as plotting the bottom row triggers an equalization of the color scale.

Source code in src/rs_vara_coordsets.m

function [vara_stats,aux_out]=rs_vara_coordsets(data_in,groupings,aux)
% [vara_stats,aux_out]=rs_vara_coordsets(data_in,groupings,aux) does an analysis of variance across groups of `dataset structures` via shuffling by trials, and provides and displays statistics
%
% Basic strategy:  records are grouped, consensus is found within each group, and the variance around the consensus is computed.
% Then, records are shuffled between groups. The shuffling can also be constrained by assigning tags to each dataset, e.g., according to the subject ID, and only shuffling within records that have the same tag.
% For each shuffle, the variance around the consensus within each group is again calculated, and compared tothat of the original data.
%
% Note that in contrast to `rs_knit_coordsets`, in which shuffling permutes the coordinates of the stimuli within a record, here the shuffling permutes the records across groups.
%
% Global consensus and consensus of groups are not returned; these can be computed with `rs_knit_coordsets`.
%
% Args:
%   data_in (struct): `dataset structure` to be processed, with fields
%
%     - ds (cell array): `coordinate structure`, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
%     - sas (cell array): `stimulus metadata structure`, sas{k} is the stimulus metadata for the kth record
%     - sets (cell array): `set metadata structure`, sets{k} is the response metadata for the kth record
%
%   groupings (struct):  structure specifying assignment of records to groups, and, optionally, tags to restrict shuffling
%     
%     - gps (int 1-D array): gps(k) is the group assignment of record k, an integer from 1 to the number of groups (ngps);  All groups must have at least one record.
%     - tags (int 1-D array): tags(k) is the tag assigned to record k.  If non-empty, shuffles will only swap a record with a record of the same tag.  Defaults to empty ([]),  
%
%   aux (struct): auxiliary inputs, may be omitted, with fields
%
%     - opts_vara (struct): options for the variance analysis, with fields
%
%         - **Transformations**
%         - allow_offset (int): 1 to allow translational offset, 0 does not allow; default is 1
%         - allow_scale (int): 1 to allow scaling of each dataset into the consensus, 0 does not allow; default is 0
%         - allow_reflection (int): 1 to allow reflection, 0 does not allow; default is 1
%         - if_normscale (int): 1 to normalize consensus to size of `data_in` (determined by geometric mean of scale factors for each dataset), 0 does not, has no effect if allow_scale=0; default is allow_scale
%
%         - **Statistics and shuffles**
%         - if_stats (int): 1 to do statistics of variance explained, 0 does not; default is 1
%         - if_exhaust (int): 1 to attempt to use exhaustive set of shuffles, otherwise will use nshuffs random shuffles if number required is > nshuffs_nax; default is 1
%         - nshuffs (int): number of shuffles to use; fewer will be made if if_exhaust=1 and number needed for exhaustive list is less than nshuffs_exhaust_max; default is 500 if if_stats=1, 0 if if_stats=0; see note below regarding statistics and plots
%         - shuffs_supplied (int 2-D array): if nonempty, user-supplied shuffles to be used; if empty, shuffles will be created according to if_exhaust and nshuffs; default is empty ([]); shuffs_supplied(ishuff,k) is the record used in place of original record k on shuffle ishuff
%         - nshuffs_max (int): maximum number shuffles that can be generated; if if_exhaust=1 and number of exhaustive shuffles exceeds this value, random shuffles will be used instead; default is 10^4
%         - if_frozen (int): random number control for shuffles and initialization; 1 for same numbers every run, 0 for different random numbers each run, negative integer for a fixed seed each run
% 
%         - **Dimension selection**
%         - dim_max_in (int): maximum dimension of data_in.ds to use; default is maximum available across all datasets
%         - dim_list_in (int 1-D array): list of dimensions to use from data_in.ds, must be unique; default is [1:dim_max_in]
%         - dim_aug (int): number of dimensions to add for consensus datasets; default is 0; see note below regarding Procrustes consensus algorithm
%         - dim_list_out (int 1-D array): list of dimensions for consensus datasets, must be same length as dim_list_in, no less than corresponding elements of dim_list_in, and unique; default is [1:dim_list_in]+dim_aug
%
%         - **Plotting and replotting**
%         - if_plot (int): 1 to plot statistics, 0 does not; default is if_stats
%         - shuff_quantiles (float 1-D array): quantiles to plot; default is [0.01 0.05 0.5 0.95 0.99]
%         - if_remove_path_label (int): 1 to remove path from filename when used as a label (in data_in.sets{:}.label), 0 does not; default is 1
%         - vara_stats (struct): include to replot a previous analysis, otherwise omit; see note below regarding replotting
%         - vara_stats_setup (struct): include to replot a previous analysis, otherwise omit; see note below regarding replotting
%
%         - **Logging and optimization**
%         - if_log (int): 1 to log progress, 0 to suppress; default is 1
%         - pcon_init_method (int or char): typically omitted; default is 0, leading to 'pca' method for initializing `procrustes_consensus`; see note below regarding Procrustes consensus algorithm
%         - if_initpca_rot (int): typically omitted, default is 1 unless any of dim_list_out>dim_list_in; see note below regarding Procrustes consensus algorithm
%         - max_iters (int): maximum number of iterations for Procrustes consensus; default is 1000; see note below regarding Procrustes consensus algorithm
%         - max_rmstol (int): maximum change ofcoordinates for consensus solution; default is 10^-5; see note below regarding Procrustes consensus algorithm
%         - pcon_initial_guess (cell array): specified initial guess for Proccrustes minimization, typically omitted; see note below regarding Procrustes consensus algorithm
%         - pcon_alignment (cell array): specified alignment for Procrustes minimization, typically omitted; see note below regarding Procrustes consensus algorithm
%         default is 1; see notes below regarding statistics and Procrustes consensus algorithm
%
%     - opts_check (struct): options for consistency checking, with field
%
%         - if_warn (int): 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
%
%     - opts_align (struct): options for alignment of data, typically omitted
%
%     - sa_pooled (struct): include to avoid recalculation of alignment, otherwise omit; see note below regarding recalculation of alignment
%     - data_align (struct): include to avoid recalculation of align ment, otherwise omit; see note below regarding recalculation of alignment
% 
% Returns:
%   vara_stats(struct): statistics and analysis parameters, detailed in note below regarding statistics and plots
% 
%   aux_out (struct): auxiliary outputs and parameter values used
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_vara (struct): aux.opts_vara, with defaults filled in
%     - opts_check (struct): aux.opts_check, with defaults filled in
%     - opts_pcon (cell array of struct): opts_pcon{idim} are the options used in Procrustes alignment for model dimension idim
%     - opts_align (struct): aux.opts_align, with defaults filled in
%     - vara_stats_setup (struct): parameters extracted from aux.opts_vara, along with the additional fields below; see note below regarding replotting a previous analysis
%
%         - nsets (int): number of records in `data_in`
%         - nstims (int): number of stimuli
%         - dataset_labels (cell array of char): dataset labels, from data_in.sets{:}.label
%         - stimulus_labels (cell array of char): stimulus labels, from data_out.sas{1}.typenames
%
%     - fig_handle (handle): handle to figure created (present only if statistics are plotted)
%
%
% Note: General notes
%     - For all records with data_in.sets{k}.type='data', the strings in data_in.sets{k}.paradigm_type must agree.
%
% Note: Note regarding statistics and plots
%     - If aux.opts_vara.if_stats=1, statistics regarding variances between individual sets and within-group consensus, and between individual sets and the global consensus, are returned in vara_stats, along with analysis parameters
%     - vara_stats is a structure with the following fields (fields names ending with _shuff will be omitted if nshuffs=0; rmsd=root-mean-squared deviation):
%         - groupings (struct): grouping information
%
%             - ngps (int): number of groups
%             - gps (int 1-D array): gps(k) is the group assignment of record k, an integer from 1 to ngps
%             - gp_list (cell array): gp_list{igp} are the indices of the records in group igp
%             - nsets_gp (int 1-D array): nsets_gp(igp) is the number of records in group igp
%             - nsets_gp_max (int): maximum size of a group
%             - tags (int 1-D array): tags(k) is the tag for record k
%
%         - nshuffs (int): number of shuffles used
%         - shuffs (int 2-D array): shuffles used; shuffs(ishuff,k) is the record used in place of original record k on shuffle ishuff
%         - opts_multi_used (struct): options used for generating shuffles, see `multi_shuff_groups` for details
%         - rmsdev_setwise (float 2-D array): rmsdev_setwise(ip,irec) is the rmsd of coordinates in record irec from the global consensus based on data_in.ds{:}{ip}, across all stimuli
%         - rmsdev_stmwise (float 2-D array): rmsdev_stmwise(ip,istm) is the rmsd of coordinates for stimulus istm from the global consensus based on data_in.ds{:}{ip}, across all records
%         - rmsdev_overall (float 2-D array): rmsdev_overall(ip,1) is the rmsd of coordinates from the global consensus based on data_in.ds{:}{ip}, across all records and stimuli
%         - rmsdev_setwise_gp (float 3-D array): rmsdev_setwise_gp(ip,irec,igp) is the rmsd of coordinates in group igp in record irec from its within-group consensus, based on data_in.ds{:}{ip}, across all stimuli
%         - rmsdev_stmwise_gp (float 3-D array): rmsdev_stmwise_gp(ip,istm,igp) is the rmsd of coordinates in group igp for stimulus istm from its within-group consensus, based on data_in.ds{:}{ip}, across all records
%         - rmsdev_overall_gp (float 3-D array): rmsdev_overall_gp(ip,1,igp) is the rmsd of coordinates in group igp from the within-group consensus, based on data_in.ds{:}{ip}, across all records and stimuli
%         - rmsdev_grpwise (float 3-D array): rmsdev_grpwise(ip,1,1) is the rmsd of coordinates from the within-group consensus, across all records and stimuli, averaged over all groups
%
%     - The counts for each of these calculations are counts_[overall|setwise|stmwise] and counts_gp_[overall|setwise|stmwise], with first array coordinate fixed at 1 (since dimension does not matter)
%     - The available rmsd (from zero) is given by rmsavail_[overall|setwise|stmwise].
%     - If aux_out.vara_stats.nshuffs>0, then a parallel computation is done after random shuffles of the stimulus labels within each record, and the results are returned in rmsdev_[overall|setwise|stimwise]_gp_shuff and rmsdev_grpwise_shuff.
%     For the shuffled quantities, the first three dimensions are the same as for the unshuffled quantities; and dimension 4 (length: nshuffs) is which shuffle.
%     The latter is the key quantity: rmsdev_grpwise_shuff(:,1,1,ishuff), to be compared with the unshuffled values, rmsdev_grpwise, indicates whether the data are more clusted into groups than would be expected by chance.
%     - To control whether the same random number seed is used on each run, use aux.opts_vara.if_frozen (default is 1).
%     - if aux.opts_vara.if_plot=1 (default if if_stats=1), then a figure is created, with three panels:
%
%         - a heatmap of rmsdev_setwise as a function of dimension
%         - a heatmap of rmsdev_setwise_gp as a function of dimension
%         - a comparison of the rmsd from the group consensus (rmsdev_grpwise, black), overall available rmsd (blue), and quantiles (rmsdev_grpwise_shuff, red) of the rmsd from the group consensus in the shuffled data
%         (quantiles are specified by shuff_quantiles; if nshuffs=0, shuffled values will not be plotted)
%
%     
% Note: Note regarding Procrustes consensus algorithm
%     - To find a consensus set of coordinates, the coordinates in each record of `data_in` are rotated, and optionally translated (if allow_offset=1),
%     scaled (if allow_scale=1), and reflected (if allow_reflection=1). These transformations are carried out for separately for each set dimension
%     for which coordinates are present in all of the records, i.e., for which data_in.ds{k}{idim} exists for all k.
%     - The algorithm, in procrustes_consensus.m, is iterative.  Briefly, after an initial guess is determined, a Procrustes 
%     transformation is found that minimizes the rms deviation between each record and the current guess. The guess is then
%     revised by setting each stimulus' coordinates equal to the centroid of the coordinates of that stimulus across the records.  To avoid drift of the updated guess, it is Procrustes-transformed for closest match
%     to an alignment coordinate set (the alignment set, unless otherwise specified, is equal to the initial guess).
%     - The iteration ends when either the number of iterations exceeds max_niters (default=1000),
%     or the rms change of the guess is less than max_rmstol (default=10^-5)
%     - There are several choices for initialization and alignment.
%
%         - For most purposes, the default initialization method (pcon_init_method=0) can be used, which uses the principal components of all the stimulus coordinates in all of the records.
%         These can be optionally forced to be centered (pcon_init_method=-1) or not (pcon_init_method=-2); if unspecified (default), centering is determined by allow_offset.
%         For these choices, if_initpca_rot=1 rotates the initial guess to match the data, or
%         not. The default for if_initpca_rot is 1 unless any of dim_list_out>dim_list_in, in which case it is 0.
%         The heuristic for not rotating if dim_list_out>dim_list_in, i.e., two or more sets of coordinates are to be knit together to construct a coordinate set with a greater number of dimensions,
%         is that without rotation, the principal components reflect projections of the coordinates that are present in any of the records.
%         -  Alternatively, pcon_init_method=r, r>0, specifies that the coordinates in data_in{r}{idim} are used.
%         -  If pcon_init_method='specify', then pcon_initial_guess{idim} is an array of size [npts ids] for the
%         initial guess, and pcon_alignment{idim}, which defaults to
%         pcon_initial_guess, is used for the alignment at the end of each iteration.  pcon_initial_guess and pcon_alignment may be omitted, in which case random values are used.
%         To control whether the same random number seed is used on each run, use if_frozen (default is 1, which uses the same random numbers on each run).
%         - The solution is only unique up to rotation (and translation and reflection, if these components are allowed).  The ambiguity is resolved by
%         matching the consensus solution to the initial guess (or, pcon_alignment{idim} if separately supplied with pcon_init_method=0), as described above.
%         - Under some circumstances (e.g., several solutions that are nearly equally good), the solution found by the algorithm may depend on
%         the initialization choice.  A simple strategy to check for this is to compare the results with the default pcon_init_method=0 to the results with
%         pcon_init_method='specify' and if_frozen=0. There are two ways that this dependency can happen.
%
%             - One is that the number of overlapping stimuli is too small. For example,
%             at least m points are required to determine a rotation and translation in an m-dimensional space; if there are fewer overlaps, then a consensus will
%             still be found but there are many other consensus datasets that fit equally well.
%             - A second way is that there are a sufficient number of points, but there are several solutions that are approximately equally good. 
%             Under these circumstances, the algorithm may get stuck in a local minimum. This possibility only occurs when there are at least three records in `data_in`, as the procedure reduces to
%             the standard Procrustes algorithm, which finds the consensus when there are only two records, is deterministic other than does rotational ambiguity.
% 
% Note: Note regarding replotting a previous analysis
%     - To replot a previous calculation with additional customization or to make a composite figure, proceed as follows:
%
%          - `data_in` and `groupings` should be equal to that used in the previous calculation.
%          - aux.vara_stats should be equal to vara_stats from the previous calculation
%          - aux.vara_stats_setup should be equal to aux_out.vara_stats_setup from the previous calculation
%
%     - The following modifications are allowed in fields of vara_stats_setup:
%
%         - dataset_labels (cell array of char): dataset labels; default is data_in.sets{:}.label
%         - stimulus_labels (cell array of char): stimulus labbels; default is data_out.sas{1}.typenames
%         - shuff_quantiles (float 1-d array): quantiles to show; default is 0.01 0.05 0.5 0.95 0.99
%         - fig_handle (handle): figure handle to plot into; figure will be created if not supplied
%         - row (int): row to plot into; default is 1
%         - nrows (int): number of rows in the figure; default is row
%
%     -  No further calculations are done
%     -  On return, vara_stats will be empty and aux_out.fig_handle will be the figure handle
%     -  In creating a composite figure, rows should be plotted in order from top to bottom, as plotting the bottom row triggers an equalization of the color scale.
%
% See also:
%   RS_ALIGN_COORDSETS, RS_AUX_CUSTOMIZE, RS_CHECK_COORDSETS, MULTI_SHUFF_GROUPS.
%
if (nargin<=2)
    aux=struct;
end
vara_stats=struct;
aux_out=struct;
%
aux=filldefault(aux,'opts_vara',struct);
aux.opts_vara=filldefault(aux.opts_vara,'if_log',1);
aux.opts_vara=filldefault(aux.opts_vara,'allow_reflection',1);
aux.opts_vara=filldefault(aux.opts_vara,'allow_offset',1);
aux.opts_vara=filldefault(aux.opts_vara,'allow_scale',0);
aux.opts_vara=filldefault(aux.opts_vara,'if_normscale',aux.opts_vara.allow_scale);
aux.opts_vara=filldefault(aux.opts_vara,'if_stats',1);
aux.opts_vara=filldefault(aux.opts_vara,'if_plot',aux.opts_vara.if_stats);
aux.opts_vara=filldefault(aux.opts_vara,'max_niters',1000);
aux.opts_vara=filldefault(aux.opts_vara,'max_rmstol',10^-5);
aux.opts_vara=filldefault(aux.opts_vara,'pcon_init_method',0);
aux.opts_vara=filldefault(aux.opts_vara,'keep_details',0);
aux.opts_vara=filldefault(aux.opts_vara,'pcon_initial_guess',[]);
aux.opts_vara=filldefault(aux.opts_vara,'pcon_alignment',aux.opts_vara.pcon_initial_guess);
aux.opts_vara=filldefault(aux.opts_vara,'if_frozen',1);
aux.opts_vara=filldefault(aux.opts_vara,'if_remove_path_label',1);
if aux.opts_vara.if_stats
    aux.opts_vara=filldefault(aux.opts_vara,'nshuffs',500);
else
    aux.opts_vara=filldefault(aux.opts_vara,'nshuffs',0);
end
aux.opts_vara=filldefault(aux.opts_vara,'shuffs_supplied',[]);
aux.opts_vara=filldefault(aux.opts_vara,'if_exhaust',1);
aux.opts_vara=filldefault(aux.opts_vara,'shuff_quantiles',[0.01 0.05 0.5 0.95 0.99]);
aux.opts_vara=filldefault(aux.opts_vara,'nshuffs_max',10^4);
%
aux=filldefault(aux,'opts_check',struct);
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
aux=filldefault(aux,'opts_pcon',struct);
%
aux=filldefault(aux,'opts_align',struct);
%
aux=rs_aux_customize(aux,'rs_vara_coordsets');
%
vara_stats=struct;
aux_out=struct;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
% replot mode
%
if isfield(aux,'vara_stats') & isfield(aux,'vara_stats_setup')
    vara_stats_setup_use=aux.vara_stats_setup;
    if isfield(vara_stats_setup_use,'fig_handle') %psg_vara_stats_plot expects figure handle in figh
        vara_stats_setup_use.figh=vara_stats_setup_use.fig_handle;
    end
    aux_out.fig_handle=psg_vara_stats_plot(aux.vara_stats,vara_stats_setup_use);
    return
end
%
%check group information
%
nsets=length(data_in.sets);
if length(groupings.gps)~=nsets
    wmsg=sprintf('group assignment list length (%2.0f) does not match number of records (%2.0f)',length(groupings.gps),nsets);
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
end
if any(~ismember(groupings.gps,[1:nsets]))
    wmsg=sprintf('group assignment list has elements not in [1:%2.0f]',nsets);
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
end
groupings.ngps=max(groupings.gps);
groupings.gp_list=cell(1,groupings.ngps);
groupings.nsets_gp=zeros(1,groupings.ngps);
for igp=1:groupings.ngps
    groupings.gp_list{igp}=find(groupings.gps==igp);
    groupings.nsets_gp(igp)=length(groupings.gp_list{igp});
end
groupings.nsets_gp_max=max(groupings.nsets_gp);
if any(groupings.nsets_gp==0)
    wmsg=sprintf('at least one group has no records');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
end
%
%check tag information
%
groupings=filldefault(groupings,'tags',[]);
if ~isempty(groupings.tags)
    if length(groupings.tags)~=nsets
        wmsg=sprintf('tag list length (%2.0f) does not match number of records (%2.0f)',length(groupings.tags),nsets);
        aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
    end
    if any(~ismember(groupings.tags,[1:max(groupings.tags)]))
        wmsg=sprintf('tag list has non-integer or non-positive values');
        aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
    end
end
%
%show setup
%
if aux.opts_vara.if_log
    disp(' ');
    disp('variance analysis');
    for iset=1:nsets
        if ~isempty(groupings.tags)
            disp(sprintf('dataset %2.0f (group %2.0f, tag %2.0f): %s',iset,groupings.gps(iset),groupings.tags(iset),data_in.sets{iset}.label));
        else
            disp(sprintf('dataset %2.0f (group %2.0f): %s',iset,groupings.gps(iset),data_in.sets{iset}.label));
        end
    end
end
%
%set up random number generator
%
if_frozen=aux.opts_vara.if_frozen;
if (if_frozen~=0) 
    rng('default');
    if (if_frozen<0)
        rand(1,abs(if_frozen));
    end
else
    rng('shuffle');
end
%
%create shuffle list via multi_shuff_groups or use supplied shuffles
%
if aux.opts_vara.nshuffs>0
    if isempty(aux.opts_vara.shuffs_supplied)
        opts_multi=struct();
        opts_multi.if_log=0;
        opts_multi.if_exhaust=aux.opts_vara.if_exhaust;
        opts_multi.if_ask=0; %non-interactive
        opts_multi.if_reduce=1; %shuffles that differ only by group order are not included
        opts_multi.if_justcount=0; %don't just count
        opts_multi.if_nowarn=1;
        opts_multi.nshuffs=min(aux.opts_vara.nshuffs,aux.opts_vara.nshuffs_max);
        opts_multi.exhaust_raw_max=aux.opts_vara.nshuffs_max;
        opts_multi.exhaust_reduced_max=aux.opts_vara.nshuffs_max;
        opts_multi.nshuffs_max=aux.opts_vara.nshuffs_max;
        opts_multi.tags=groupings.tags;
        %
        [shuffs,gp_info,opts_multi_used]=multi_shuff_groups(groupings.gps,opts_multi);
        if aux.opts_vara.if_log
            disp(sprintf('shuffles made:  %6.0f, requested:%6.0f',size(shuffs,1),aux.opts_vara.nshuffs));
            disp(sprintf('exhautive mode used: %1.0f, requested:     %1.0f',opts_multi_used.if_exhaust,aux.opts_vara.if_exhaust));
        end
        aux.opts_vara.if_exhaust=opts_multi_used.if_exhaust;
    else %user-supplied shuffles
        shuffs=aux.opts_vara.shuffs_supplied;
        opts_multi_used=struct();
        if  size(shuffs,1)~=aux.opts_vara.nshuffs
            wmsg=sprintf('number of supplied shuffles (%6.0f) does not match number of shuffles requested (%6.0f)',size(shuffs,1),aux.opts_vara.nshuffs);
            aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
        end
        if  size(shuffs,2)~=nsets
            wmsg=sprintf('number of elements in supplied shuffles (%6.0f) does not match number of records (%6.0f)',size(shuffs,2),nsets);
            aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
        end
        no_perm=0;
        for ishuff=1:size(shuffs,1)
            if any(sort(shuffs(ishuff,:))~=[1:size(shuffs,2)])
                no_perm=no_perm+1;
            end
        end
        if no_perm>0
            wmsg=sprintf('supplied shuffles are illegal: %3.0f fail to be permutations of the number of records',no_perm);
            aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
        end
        if aux.opts_vara.if_log
            disp(sprintf('shuffles supplied:  %6.0f, requested:%6.0f',size(shuffs,1),aux.opts_vara.nshuffs));
        end
    end
else
    shuffs=[];
    opts_multi_used=struct;
end
vara_stats.shuffs=shuffs;
vara_stats.nshuffs=size(shuffs,1);
vara_stats.opts_multi_used=opts_multi_used;
aux.opts_vara.nshuffs=vara_stats.nshuffs;
%
%
% tally missing stimuli in input datasets and align according to all stimuli
%
nstims_each=zeros(1,nsets);
stims_nan=cell(1,nsets);
if aux.opts_vara.if_log
    disp('before alignment of stimuli')
end
for iset=1:nsets
    nstims_each(iset)=data_in.sas{iset}.nstims;
    stims_nan{iset}=find(isnan(data_in.ds{iset}{1}));
    disp(sprintf('set %2.0f: %2.0f stimuli (%2.0f are NaN), label: %s',iset,nstims_each(iset),length(stims_nan{iset}),data_in.sets{iset}.label))
end
%
%align and check consistency
%
aux2=aux;
aux2.opts_align.if_log=aux.opts_vara.if_log;
[data_align,aux_align]=rs_align_coordsets(data_in,aux2);
%
%check consistency of data files
%
check=rs_check_coordsets(data_align,aux.opts_check);
if ~isempty(check.warnings)
    aux_out.warnings=strvcat(aux_out.warnings,check.warnings); %kludge since strvcat([],[])='', but we want [] if both warnings are empty
end
aux_out.warn_bad=aux_out.warn_bad+check.warn_bad;
%
nstims_each=check.nstims_each;
dim_list_each=check.dim_list_each;
dim_list_union=check.dim_list_union;
dim_list_inter=check.dim_list_inter;
typenames_each=check.typenames_each;
typenames_union=check.typenames_union;
typenames_inter=check.typenames_inter;
%
if min(nstims_each)~=max(nstims_each)
    disp('cannot proceed');
    disp(aux_out.warnings);
    return
end
nstims=min(nstims_each);
if aux.opts_vara.if_log
    disp('after alignment of stimuli')
    disp(sprintf('total stimuli: %3.0f',nstims));
end
%
%set up dimension defaults
%
aux.opts_vara=filldefault(aux.opts_vara,'dim_max_in',max(dim_list_inter));
aux.opts_vara=filldefault(aux.opts_vara,'dim_list_in',[1:aux.opts_vara.dim_max_in]);
aux.opts_vara=filldefault(aux.opts_vara,'dim_aug',0);
aux.opts_vara=filldefault(aux.opts_vara,'dim_list_out',aux.opts_vara.dim_aug+aux.opts_vara.dim_list_in);
if length(aux.opts_vara.dim_list_in)~=length(aux.opts_vara.dim_list_out)
    wmsg=sprintf('dim_list_in and dim_list_out have different lengths');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
else
    if_aug=any(aux.opts_vara.dim_list_out>aux.opts_vara.dim_list_in);
    aux.opts_vara=filldefault(aux.opts_vara,'if_initpca_rot',1-if_aug);
    %
    if any(aux.opts_vara.dim_list_out<aux.opts_vara.dim_list_in)
        wmsg=sprintf('dim_list_in values cannot be more than dim_list_out values');
        aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
    end
end
if length(aux.opts_vara.dim_list_in)~=length(unique(aux.opts_vara.dim_list_in))
    wmsg=sprintf('dim_list_in values are not unique');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
end
if length(aux.opts_vara.dim_list_out)~=length(unique(aux.opts_vara.dim_list_out))
    wmsg=sprintf('dim_list_out values are not unique');
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',1));
end
%
if ischar(aux.opts_vara.pcon_init_method)
    if strcmp(aux.opts_vara.pcon_init_method,'specify')
        aux.opts_vara.initialize_set=0; %opts_vara.pcon_initial_guess and opts_vara.pcon_alignment will be used
    else
        wmsg='initialization method not recognized; default used';
        aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_check.if_warn));
        aux.opts_vara.pcon_init_method=0;
        aux.opts_vara.initialize_set='pca';
    end
else
    if aux.opts_vara.pcon_init_method>0
        aux.opts_vara.initialize_set=aux.opts_vara.pcon_init_method;
    elseif aux.opts_vara.pcon_init_method==0
        aux.opts_vara.initialize_set='pca';
    elseif aux.opts_vara.pcon_init_method==-1
        aux.opts_vara.initialize_set='pca_center';
    else
        aux.opts_vara.initialize_set='pca_nocenter';
    end
end
%
%reformat for consensus calculations
%
pcon_dim_max_in=max(dim_list_inter);
pcon_dim_max_out=max(aux.opts_vara.dim_list_out);
z=cell(pcon_dim_max_in,1);
for ip=1:pcon_dim_max_in
    if ismember(ip,dim_list_inter)
        z{ip}=zeros(nstims,ip,nsets);
        for iset=1:nsets
            z{ip}(:,:,iset)=data_align.ds{iset}{ip}(:,[1:ip]); %only include data up to pcon_dim_use
            z{ip}(aux_align.opts_align.which_common(:,iset)==0,:,iset)=NaN; % pad with NaN's if no data
        end
    end
end
%
%overlaps indicates same stimulus (from ovlp_array) and also that the coordinates are not NaN's
coords_isnan=reshape(isnan(z{1}),[nstims,nsets]);
opts_pcon.overlaps=aux_align.ovlp_array.*(1-coords_isnan);
if aux.opts_vara.if_log
    disp(sprintf('number of overlapping stimuli in component removed because coordinates are NaN'));
    disp(sum(coords_isnan.*aux_align.ovlp_array,1));
    disp('overlap matrix from stimulus matches, without regard as to whether stimulus coordinates coordinates are NaN')
    disp(aux_align.ovlp_array'*aux_align.ovlp_array);
    disp('overlap matrix from stimulus matches, but excluding stimuli for which coordinates that are NaN')
    disp(opts_pcon.overlaps'*opts_pcon.overlaps);
end
%
%define results structures
%
results=struct;
consensus=cell(pcon_dim_max_out,1); %d1: dimnension
znew=cell(pcon_dim_max_out,1);
opts_pcon_used=cell(pcon_dim_max_out,1);
%
nshuffs=vara_stats.nshuffs;
nsets_gp=groupings.nsets_gp;
nsets_gp_max=groupings.nsets_gp_max;
ngps=groupings.ngps;
%
%these are vector distances, taking all coordinates into account
%note that the "allow scale" coordinate in psg_align_vara_demo is omitted,
%
rmsdev_setwise=zeros(pcon_dim_max_out,nsets); %d1: dimension, d2: set
rmsdev_stmwise=zeros(pcon_dim_max_out,nstims); %d1: dimension, d2: stim
rmsdev_overall=zeros(pcon_dim_max_out,1); %rms distance, across all datasets and stimuli
%
rmsdev_setwise_gp=zeros(pcon_dim_max_out,nsets_gp_max,ngps); %d1: dimension, d2: set (within group), d3: gp
rmsdev_stmwise_gp=zeros(pcon_dim_max_out,nstims,ngps); %d1: dimension, d2: stim, d3: gp
rmsdev_overall_gp=zeros(pcon_dim_max_out,1,ngps); %d1: dimension, d2: dummy, d3: gp
%
counts_setwise=zeros(1,nsets);
counts_stmwise=zeros(1,nstims);
counts_overall=zeros(1);
%
counts_setwise_gp=zeros(1,nsets_gp_max,ngps);
counts_stmwise_gp=zeros(1,nstims,ngps);
counts_overall_gp=zeros(1,1,ngps);
%
%for setwise and stmwise, use NaN so that missing data won't affect averages
rmsdev_setwise_gp_shuff=NaN(pcon_dim_max_out,nsets_gp_max,ngps,nshuffs); %d1: dimension, d2: set, d3: gp, d4: shuffle
rmsdev_stmwise_gp_shuff=NaN(pcon_dim_max_out,nstims,ngps,nshuffs); %d1: dimension, d2: stim, d3: gp, d4: shuffle
rmsdev_overall_gp_shuff=zeros(pcon_dim_max_out,1,ngps,nshuffs); %d1: dimension, d3: gp, d4: shuffle
%
%rms variance available in original data
rmsavail_setwise=zeros(pcon_dim_max_out,nsets);
rmsavail_stmwise=zeros(pcon_dim_max_out,nstims);
rmsavail_overall=zeros(pcon_dim_max_out,1);
for ip=1:pcon_dim_max_out
    sqs=sum(z{ip}.^2,2);
    rmsavail_setwise(ip,:)=reshape(sqrt(mean(sqs,1,'omitnan')),[1 nsets]);
    rmsavail_stmwise(ip,:)=reshape(sqrt(mean(sqs,3,'omitnan')),[1 nstims]);
    rmsavail_overall(ip,:)=sqrt(mean(sqs(:),'omitnan'));
end
%
aux_out.opts_check=aux.opts_check;
if aux_out.warn_bad==0 %     %process
    typenames_all=typenames_inter; %because stimuli are required to be the same across datasets
    %
    for dptr=1:length(aux.opts_vara.dim_list_in)
        ip=aux.opts_vara.dim_list_in(dptr);
        if ismember(ip,dim_list_inter)
            dim_out=aux.opts_vara.dim_list_out(dptr); %find corresponding output dimension
            %now pad if necessary
            z_aug=z{ip};
            if ip<dim_out
                z_aug=cat(2,z_aug,zeros(nstims,dim_out-ip,nsets));
            end
            opts_pcon=aux.opts_vara;
            %find global consensus (independent of shuffle)
            [consensus{ip},znew{ip},ts,details,opts_pcon_used{ip}]=procrustes_consensus(z_aug,opts_pcon);
            if aux.opts_vara.if_log
                disp(sprintf(' creating global Procrustes consensus for dim %2.0f based on datasets of dim %2.0f, iterations: %4.0f, final total rms dev per coordinate: %8.5f',...
                    dim_out,ip,length(details.rms_change),sqrt(sum(details.rms_dev(:,end).^2))));               
            end
            sqdevs=sum((znew{ip}-repmat(consensus{ip},[1 1 nsets])).^2,2); %squared deviation of consensus from rotated component
            %rms deviation across each dataset, summed over coords, normalized by the number of stimuli in each dataset
            rmsdev_setwise(ip,:)=reshape(sqrt(mean(sqdevs,1,'omitnan')),[1 nsets]);
            counts_setwise=squeeze(sum(~isnan(sqdevs),1))';
            %rms deviation across each stimulus, summed over coords, normalized by the number of sets that include the stimulus
            rmsdev_stmwise(ip,:)=reshape(sqrt(mean(sqdevs,3,'omitnan')),[1 nstims]);
            counts_stmwise=(sum(~isnan(sqdevs),3))';
            %rms deviation across all stimuli and coords
            rmsdev_overall(ip,1)=sqrt(mean(sqdevs(:),'omitnan'));
            counts_overall=sum(~isnan(sqdevs(:)));
            %
            %do shuffles, shuffle 0 = unshuffled
            %
            for ishuff=0:nshuffs
                if (ishuff==0)
                    perm_use=[1:nsets];
                else
                    perm_use=shuffs(ishuff,:);
                end
                zs=z_aug(:,:,perm_use); %the datasets in permuted order, with NaN's where stimuli are missing
                for igp=1:ngps
                    zg=zeros(nstims,dim_out,nsets_gp(igp));
                    for iset_ptr=1:nsets_gp(igp)
                        iset=groupings.gp_list{igp}(iset_ptr); %a dataset in this group
                        zg(:,:,iset_ptr)=zs(:,:,iset); %the (shuffled) datasets in this group
                    end %iset_ptr
                    stims_gp=find(~all(any(isnan(zg),2),3)); %if some coord is NaN in all of the datasets
                    zg=zg(stims_gp,:,:); %keep only the stimuli that have data
                    %now form a consensus from each group
                    overlaps_gp=1-reshape(any(isnan(zg),2),[length(stims_gp),nsets_gp(igp)]); %overlaps within group
                    [consensus_gp,znew_gp,ts_gp,details_gp]=procrustes_consensus(zg,setfield(opts_pcon,'overlaps',overlaps_gp));
                    r=sqrt(sum(details_gp.rms_dev(:,end).^2));
                    %
                    sqdevs_gp=sum((znew_gp-repmat(consensus_gp,[1 1 nsets_gp(igp)])).^2,2); %squared deviation of group consensus from rotated component
                    rms_setwise_gp=reshape(sqrt(mean(sqdevs_gp,1,'omitnan')),[1 nsets_gp(igp)]);
                    rms_stmwise_gp=reshape(sqrt(mean(sqdevs_gp,3,'omitnan')),[1 length(stims_gp)]);
                    rms_overall_gp=sqrt(mean(sqdevs_gp(:),'omitnan'));
                    %
                    if (ishuff==0)
                        rmsdev_setwise_gp(ip,[1:nsets_gp(igp)],igp)=rms_setwise_gp;
                        counts_setwise_gp(1,[1:nsets_gp(igp)],igp)=squeeze(sum(~isnan(sqdevs_gp),1))';
                        %rms deviation across each stimulus, summed over coords, normalized by the number of sets that include the stimulus
                        rmsdev_stmwise_gp(ip,stims_gp,igp)=rms_stmwise_gp;
                        counts_stmwise_gp(1,stims_gp,igp)=(sum(~isnan(sqdevs_gp),3))';
                        %rms deviation across all stimuli and coords
                        rmsdev_overall_gp(ip,1,igp)=rms_overall_gp;
                        counts_overall_gp(1,1,igp)=sum(~isnan(sqdevs_gp(:)));
                        %
                   else
                        rmsdev_setwise_gp_shuff(ip,[1:nsets_gp(igp)],igp,ishuff)=rms_setwise_gp;
                        rmsdev_stmwise_gp_shuff(ip,stims_gp,igp,ishuff)=rms_stmwise_gp;
                        rmsdev_overall_gp_shuff(ip,1,igp,ishuff)=rms_overall_gp;
                    end
                    if aux.opts_vara.if_log
                        if ishuff==0
                            disp(sprintf('  grp %2.0f: %3.0f datasets, %3.0f of %3.0f stimuli, Procrustes consensus iterations: %4.0f, final total rms dev per coordinate: %8.5f',...
                                igp,nsets_gp(igp),length(stims_gp),nstims,length(details_gp.rms_change),r));
                        end
                        if (ishuff==nshuffs) & (ishuff>0)
                            disp(sprintf('  grp %2.0f: total rms vec distance in data %8.5f; in %5.0f shuffles, range: [%8.5f %8.5f]',...
                                igp,rmsdev_overall_gp(ip,1,igp),nshuffs,...
                                min(rmsdev_overall_gp_shuff(ip,1,igp,:),[],4),max(rmsdev_overall_gp_shuff(ip,1,igp,:),[],4)));
                        end
                    end %if_log
                end %igp
            end %ishuff
        end %dim is available in all datasets
    end %ip
    vara_stats.groupings=groupings;
    %
    vara_stats.rmsdev_desc='d1: dimension, d2: nsets or nstims';
    vara_stats.rmsdev_setwise=rmsdev_setwise;
    vara_stats.rmsdev_stmwise=rmsdev_stmwise;
    vara_stats.rmsdev_overall=rmsdev_overall;
    %
    vara_stats.rmsdev_gp_desc='d1: dimension, d2: nsets or nstims, d3: group';
    vara_stats.rmsdev_setwise_gp=rmsdev_setwise_gp;
    vara_stats.rmsdev_stmwise_gp=rmsdev_stmwise_gp;
    vara_stats.rmsdev_overall_gp=rmsdev_overall_gp;
    % sum of within-gp rms dev explained weighted by number of sets within each group
    vara_stats.rmsdev_grpwise=sqrt(sum(rmsdev_overall_gp.^2.*repmat(reshape(nsets_gp(:),[1 1 ngps]),[pcon_dim_max_out,1,1]),3)/nsets);
    %
    vara_stats.counts_desc='d1: 1, d2: nsets or nstims';
    vara_stats.counts_setwise=counts_setwise;
    vara_stats.counts_stmwise=counts_stmwise;
    vara_stats.counts_overall=counts_overall;
    %
    vara_stats.counts_gp_desc='d1: 1, d2: nsets or nstims, d3: group';
    vara_stats.counts_setwise_gp=counts_setwise_gp;
    vara_stats.counts_stmwise_gp=counts_stmwise_gp;
    vara_stats.counts_overall_gp=counts_overall_gp;
    %
    vara_stats.rmsavail_setwise=rmsavail_setwise;
    vara_stats.rmsavail_stmwise=rmsavail_stmwise;
    vara_stats.rmsavail_overall=rmsavail_overall;
    %
    if (nshuffs>0)
        vara_stats.rmsdev_gp_shuff_desc='d1: dimension, d2: nsets or nstims, d3: group, d4: shuffle';
        vara_stats.rmsdev_setwise_gp_shuff=rmsdev_setwise_gp_shuff;
        vara_stats.rmsdev_stmwise_gp_shuff=rmsdev_stmwise_gp_shuff;
        vara_stats.rmsdev_overall_gp_shuff=rmsdev_overall_gp_shuff;
        vara_stats.rmsdev_grpwise_shuff=sqrt(sum(rmsdev_overall_gp_shuff.^2.*repmat(reshape(nsets_gp(:),[1 1 ngps 1]),[pcon_dim_max_out,1,1,nshuffs]),3)/nsets);
    end
    %
    %plot if requested
    %
    if aux.opts_vara.if_stats
        vara_stats_setup.opts_vara=aux.opts_vara;
        vara_stats_setup.nsets=nsets;
        vara_stats_setup.dim_list_in_max=max(aux.opts_vara.dim_list_in);
        vara_stats_setup.dim_list_in=aux.opts_vara.dim_list_in;
        vara_stats_setup.dim_list_out=aux.opts_vara.dim_list_out;
        vara_stats_setup.dataset_labels=cell(1,nsets);
        for iset=1:nsets
            ds_label=data_in.sets{iset}.label;
            if aux.opts_vara.if_remove_path_label
                ds_label=ds_label(1+max(max(union(find(ds_label=='\'),find(ds_label=='/'))),0):end);
                ds_label=strrep(ds_label,'.mat','');
            end
            vara_stats_setup.dataset_labels{iset}=ds_label;
        end
        vara_stats_setup.stimulus_labels=typenames_all;
        vara_stats_setup.nshuffs=aux.opts_vara.nshuffs;
        vara_stats_setup.shuff_quantiles=aux.opts_vara.shuff_quantiles;
        vara_stats_setup.nstims=nstims;
        %
        aux_out.vara_stats_setup=vara_stats_setup;
        if aux.opts_vara.if_plot
            vara_stats_setup_use=aux_out.vara_stats_setup;
            if isfield(vara_stats_setup_use,'fig_handle')
                vara_stats_setup_use.figh=vara_stats_setup_use.fig_handle; %psg_vara_stats_plot expects figure handle in figh
            end
            aux_out.fig_handle=psg_vara_stats_plot(vara_stats,vara_stats_setup_use);
        end
    end
    %
    aux_out.opts_vara=aux.opts_vara;
    aux_out.opts_pcon=opts_pcon_used;
    aux_out.opts_align=aux_align.opts_align;
    %
else %cannot process
    aux_out.opts_vara=aux.opts_vara;
    vara_stats.groupings=groupings;
    disp('cannot proceed');
    disp(aux_out.warnings);
end
return
end

`rs_warning(msg, if_bad, opts_warn)`

opts_used=rs_warning(msg,if_bad,opts_warn) displays a warning message and updates a structure of warning tallies and messages

Input arguments:

msg (char or vertical concatenation of char) –

one or more warning messages
if_bad (int) –

severity, can =be omitted; 1 indicates that furhter processing cannot proceed, 0 is less serious; default is 1
opts_warn (struct) –
options, can be omitted, with fields
- if_warn (int) – 1 to display warning, 0 to omit; default is 1
- warnings (char or vertical concatenaton of char) – previous warnings; default is []
- warn_bad (int) – current tally of serious warnings; default is 0
- warn_leadin (char) a prefix for warnings echoed at console; default is '##### rs_warning: '; can be modified by editing rs_aux_defaults_define
- if_warn_traceback (int) – 1 to show a traceback with each warning, 0 to omit; default is 0; if 1, forces if_warn to 1

Output arguments:

opts_used ( struct ) –

updated opts_warn structure

Typical usage

Typical usage is aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux_out.if_warn)) [?how to show code?] which appends the warning message 'wmsg' to aux_out.warnings, and increments tally of aux_out.warn_bad

Source code in src/rs_warning.m

function opts_used=rs_warning(msg,if_bad,opts_warn)
%opts_used=rs_warning(msg,if_bad,opts_warn) displays a warning message and updates a structure of warning tallies and messages
% 
% Args:
%   msg (char or vertical concatenation of char): one or more warning messages
%   if_bad (int): severity, can =be omitted; 1 indicates that furhter processing cannot proceed, 0 is less serious; default is 1
%   opts_warn (struct): options, can be omitted, with fields
%
%      - if_warn (int): 1 to display warning, 0 to omit; default is 1
%      - warnings (char or vertical concatenaton of char): previous warnings; default is []
%      - warn_bad (int): current tally of serious warnings; default is 0
%      - warn_leadin (char) a prefix for warnings echoed at console;
%      default is '##### rs_warning: '; can be modified by editing `rs_aux_defaults_define`
%      - if_warn_traceback (int): 1 to show a traceback with each warning, 0 to omit; default is 0; if 1, forces if_warn to 1
%
% Returns:
%   opts_used (struct): updated opts_warn structure
%
% Note: Typical usage
%   Typical usage is
%   aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux_out.if_warn)) [?how to show code?]
%   which appends the warning message 'wmsg' to aux_out.warnings, and increments tally of aux_out.warn_bad
%
if (nargin<=1)
    if_bad=1;
end
if (nargin<=2)
    opts_warn=struct();
end
opts_warn=filldefault(opts_warn,'if_warn',1);
opts_warn=filldefault(opts_warn,'warnings',[]);
opts_warn=filldefault(opts_warn,'warn_bad',0);
opts_warn=filldefault(opts_warn,'warn_leadin',getfield(getfield(rs_aux_customize(struct()),'overall'),'warn_leadin'));
opts_warn=filldefault(opts_warn,'if_warn_traceback',getfield(getfield(rs_aux_customize(struct()),'overall'),'if_warn_traceback'));
if opts_warn.if_warn_traceback
    opts_warn.if_warn=1;
end
%
if opts_warn.if_warn
    for k=1:size(msg,1)
        disp(cat(2,opts_warn.warn_leadin,msg(k,:)));
    end
end
if opts_warn.if_warn_traceback
    warning(opts_warn.warn_leadin);
end
opts_warn.warnings=strvcat(opts_warn.warnings,msg);
opts_warn.warn_bad=opts_warn.warn_bad+if_bad;
%
opts_used=opts_warn;
return
end

`rs_write_coorddata(fullname, data_in, aux)`

aux_out=rs_write_coorddata(fullname,data_in,aux) writes a single record in a dataset structure to a file

Input arguments:

fullname (char or singleton cell array) –

file name with path, if empty, it will be requested interactively. The file name should contain the string '_coords'
data_in (struct) –
dataset structure containing the record to be written
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record
aux (struct) –
auxiliary options, with fields
- opts_write (struct) – , with fields
  - set_no (int) – record number to write; default is 1
  - if_embed (int) – 1 to embed the setup metadata in the output file, 0 to omit; see note below re setup metadata; default is 1.
  - ui_prompt (char) – User interface prompt, default ia ,'Select a coordinate file to write'
  - if_gui (int) – 1 to use graphical interface to get files if file names are not supplied; 0 to use text prompt; default is 0
  - if_log (int) – 1 to log progress, 0 to omit; default is 1
  - data_fullname_def (char) – default file name to write, used as a prompt if if_gui=0 if fullname is not provided; default is '[path and file name for the default coordinate dataset file to write].mat'; default can be changed by editing the line containing generic.opts_write.coord_data_fullname_write_def in rs_aux_defaults_define, running it once, and saving the workspace as rs_aux_defaults.mat.
  - if_uselocal (int) – should be set to 0 (default); 1 (intended for maintenance only) overrides options set by rs_aux_defaults_define by defaults in psg_localopts.m

Output arguments:

aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings
- warn_bad (int) – number of warnings that prevent further processing
- opts_write (struct) – aux, aux.opts_geof, with defaults filled in
- fullname (char) – file name written
- s_written (struct) – structure written

Note: Note re setup metadata

The setup metadata is required for binary texture domain datasets; not otherwise required unless configured on installation.
If setup metadata is required, it is read along with the coordinate data by rs_read_coorddata or rs_get_coordsets, and kept in data_in{k}.sas; it is also updated by rs_align_coordsets and rs_knit_coordsets
if_embed=1 embeds the metadata in the written file, so that the setup file no longer needs to be read.

`rs_xform_apply(data_in, xforms, aux)`

[data_out,aux_out]=rs_xform_apply(data_in,xforms,aux) applies a transformation structure to the coordinates in a dataset structure

A transformation structure is a cell array of geometric transformations. This module can apply several types of transformations: linear, linear with offset (affine), projective, piecewise affine, and piecewise projective.

Input arguments:

data_in (struct) –
dataset structure to be aligned containing n records, with fields
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record
xforms (struct) –
specification of the transformations, typically an output structure from rs_xform_specify
- ts (cell array) – ts{k}{idim} is the transformation to be applied to the coordinates of dimension idim in record k; see note below regarding transformations
- pipeline (struct) – a structure that becomes the starting point for the 'pipeline' field of a set metadata structure when the transformations are applied, may be omitted; default is omitted
aux (struct) –
auxiliary options, may be omitted, with fields
- opts_xform (struct) – specification of the transformation, with fields
  - class (char) – 'affine','mean','procrustes','projective','pwaffine','pwprojective'; default is 'affine'; see transformation structures and note below regarding transformations
  - if_warn (int) – 1 (default) to show warnings
- opts_check (struct) – options for consistency checking, with field
  - if_warn (int) – 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1

Output arguments:

data_out ( struct ) –

a dataset structure with the transformations applied, same format as as data_in
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_xform (struct) – aux.opts_xform, with defaults filled in
- opts_check (struct) – aux.opts_check, with defaults filled in

Note regarding transformations

xforms.ts{k}{idim} is the transformation to be applied to record k in data_in, i.e., to the coordinates data_in.ds{k}{idim}.
- If length(xforms.ts{k})<length(data_in), transformations are used in cyclic order.
- If any of xforms.ts{k}{idim} are missing, then the coordinates in data_in.ds{k}{idim} are passed to data_out unchanged.
Several classes of transformations, specified by aux.opts_xforms.class, are supported (see transformation structures). The parameters in xforms.ts{k}{idim} have the following meaning, where output and input both have size [nstims idim], and ts=ts{k}{idim}:
- affine: [output(istim,:)]=ts.b * [input(istim,:)] * ts.T + ts.c; where size(b)=1, size(T)=[idim,idim], and size(c)=[1 idim]. If these fields are not present and xform.class='affine', then alternative parameter names are allowed: 'scaling' for b, 'orthog' for T, and 'translation' for c. This allows for compatibility with the transformations produced by procrustes_consensus.
- procrustes: same as affine, but ts.b=1 and abs(det(T))=1. These are not checked.
- mean: same as affine, but ts.b=1, T=0. These are not checked.
- projective: affine parameters and also p, size [isim 1]. p=0 reduces to affine. See transformation structures for further details.
- pwaffine (piecewise affine): b as in affine. T has size [idim idim 2^ncuts] and c has size [2^ncuts idim], specifying the affine transformation on each cut. vcut has size [ncuts idim], each row is a unit vector, orthogonal to the cutplanes. acut has size [1 2^ncuts], specifying the cutpoints. See transformation structures for further details.
- pwprojective (piecewise projective): parameters as in pwaffine, and also p, of size [idim, 2^ncuts], used as in projective for each component See transformation structures for further details.

See also: rs_aux_customize, rs_check_coordsets, rs_xform_specify, RS_FORM_SPECIFY_APPLY_TEST, PSG_GEOMODELS_APPLY, procrustes_compat.

Source code in src/rs_xform_apply.m

function [data_out,aux_out]=rs_xform_apply(data_in,xforms,aux)
% [data_out,aux_out]=rs_xform_apply(data_in,xforms,aux)
% applies a `transformation structure` to the coordinates in a `dataset structure`
%
% A `transformation structure` is a cell array of geometric transformations.
% This module can apply several types of transformations: linear, linear with offset (affine), projective, piecewise affine, and piecewise projective.
%
% Args:
%   data_in (struct): `dataset structure` to be aligned containing n records, with fields
%
%     - ds (cell array): `coordinate structure`, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
%     - sas (cell array): `stimulus metadata structure`, sas{k} is the stimulus metadata for the kth record
%     - sets (cell array): `set metadata structure`, sets{k} is the response metadata for the kth record
%
%   xforms (struct): specification of the transformations, typically an output structure from `rs_xform_specify`
% 
%      - ts (cell array): ts{k}{idim} is the transformation to be applied to the coordinates of dimension idim in record k; see note below regarding transformations
%      - pipeline (struct): a structure that becomes the starting point for
%      the 'pipeline' field of a `set metadata structure` when the transformations are applied, may be omitted; default is omitted
%
%   aux (struct): auxiliary options, may be omitted, with fields
%
%     - opts_xform (struct): specification of the transformation, with fields
%
%         - class (char): 'affine','mean','procrustes','projective','pwaffine','pwprojective'; default is 'affine'; see `transformation structures` and note below regarding transformations
%         - if_warn (int): 1 (default) to show warnings
%
%     - opts_check (struct): options for consistency checking, with field
%
%          - if_warn (int): 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
%
% Returns:
%   data_out (struct): a `dataset structure` with the transformations applied, same format as  as `data_in`
%
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_xform (struct): aux.opts_xform, with defaults filled in
%     - opts_check (struct): aux.opts_check, with defaults filled in
%
% Note: Note regarding transformations
%   - xforms.ts{k}{idim} is the transformation to be applied to record k in `data_in`, i.e., to the coordinates data_in.ds{k}{idim}.
%
%       - If length(xforms.ts{k})<length(data_in), transformations are used in cyclic order.
%       - If any of xforms.ts{k}{idim} are missing, then the coordinates in data_in.ds{k}{idim} are passed to `data_out` unchanged.
%
%    - Several classes of transformations, specified by aux.opts_xforms.class, are supported (see `transformation structures`).  The parameters in
%    xforms.ts{k}{idim} have the following meaning, where output and input both have size [nstims idim], and ts=ts{k}{idim}:
%
%       - affine: [output(istim,:)]=ts.b * [input(istim,:)] * ts.T + ts.c;
%       where size(b)=1, size(T)=[idim,idim], and size(c)=[1 idim]. If these fields are not present and xform.class='affine', then
%       alternative parameter names are allowed: 'scaling' for b, 'orthog' for T, and 'translation' for c.
%       This allows for compatibility with the transformations produced by `procrustes_consensus`.
%       - procrustes: same as affine, but ts.b=1 and abs(det(T))=1.  *These are not checked.*
%       - mean: same as affine, but ts.b=1, T=0. *These are not checked.*
%       - projective: affine parameters and also p, size [isim 1]. p=0
%       reduces to affine.  See `transformation structures` for further details.
%       - pwaffine (piecewise affine): b as in affine.  T has size [idim idim 2^ncuts] and c
%       has size [2^ncuts idim], specifying the affine transformation on
%       each cut. vcut has size [ncuts idim], each row is a unit vector,
%       orthogonal to the cutplanes. acut has size [1 2^ncuts], specifying the cutpoints.
%       See `transformation structures` for further details.
%       - pwprojective (piecewise projective): parameters as in pwaffine, and also p, of size [idim, 2^ncuts], used as in projective for each component
%       See `transformation structures` for further details. 
%
%  See also: RS_AUX_CUSTOMIZE, RS_CHECK_COORDSETS, RS_XFORM_SPECIFY, RS_FORM_SPECIFY_APPLY_TEST,
%  PSG_GEOMODELS_APPLY, PROCRUSTES_COMPAT.
%
if (nargin<=2)
    aux=struct;
end
%
class_default='affine';
aux=filldefault(aux,'opts_xform',struct); 
aux.opts_xform=filldefault(aux.opts_xform,'if_warn',1);
aux.opts_xform=filldefault(aux.opts_xform,'class',class_default);
%
aux=filldefault(aux,'opts_check',struct);
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
aux=rs_aux_customize(aux,'rs_xform_apply');
%
aux_out=struct;
data_out=struct;
%
%check consistency and get available stimuli, dimensions, typenames
%
check=rs_check_coordsets(data_in,aux.opts_check);
%
aux_out.warnings=check.warnings;
aux_out.warn_bad=check.warn_bad;
nsets=check.nsets;
nstims_each=check.nstims_each;
dim_list_each=check.dim_list_each;
dim_list_union=check.dim_list_union;
dim_list_inter=check.dim_list_inter;
typenames_each=check.typenames_each;
typenames_union=check.typenames_union;
typenames_inter=check.typenames_inter;
%
params_needed=struct;
params_needed.mean={'b','T','c'};
params_needed.procrustes={'b','T','c'};
params_needed.affine={'b','T','c'};
params_needed.projective={'b','T','c','p'};
params_needed.pwaffine={'b','T','c','vcut','acut'};
params_needed.pwprojective={'b','T','c','vcut','acut','p'};
%
missing_params=[];
%
%check number of transform sets
%
nsets_xform=length(xforms.ts);
if (nsets_xform~=nsets) & (nsets_xform~=1)
    wmsg=sprintf('number of transform sets specified (%2.0f) differs from number of datasets (%2.0f)',nsets_xform,nsets);
    aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_xform.if_warn));
end
data_out.ds=cell(1,nsets);
data_out.sas=cell(1,nsets);
data_out.sets=cell(1,nsets);
%
class=aux.opts_xform.class;
if ~isfield(params_needed,class)
    wmsg=sprintf('transformation class %s not recognized, set to %s',class,class_default);
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_xform.if_warn));
    aux.opts_xform.class=class_default;
    class=class_default;
end
for k=1:nsets
    k_xform=mod(k-1,nsets_xform)+1;
    dim_list_out=[];
    data_out.ds{k}=cell(1,0);
    for idim=1:length(data_in.ds{k})
        coords=data_in.ds{k}{idim};
        have_xform=0;
        if idim<=length(xforms.ts{k_xform})
            if isstruct(xforms.ts{k_xform}{idim})
                have_xform=1;
            end
        end
        if have_xform %check that parameters are present and reformat if needed
            ts=xforms.ts{k_xform}{idim};
            if strcmp(aux.opts_xform.class,'affine') %check to see if alternate names are used
                if ~isempty(ts)
                    if ((~isfield(ts,'b') | ~isfield(ts,'T') | ~isfield(ts,'c')) & (isfield(ts,'scaling') & isfield(ts,'orthog') & isfield(ts,'translation')))
                        ts=procrustes_compat(ts);
                    end
                end
            end
            if isempty(ts)
                ts=struct;
            end
            for ifn=1:length(params_needed.(class))
                if ~isfield(ts,params_needed.(class){ifn})
                    missing_params=strvcat(missing_params,params_needed.(class){ifn});
                    have_xform=0;
                end
            end
        end 
        if have_xform==0 | isempty(coords)
            coords_new=coords;
        else
            coords_new=psg_geomodels_apply(aux.opts_xform.class,coords,ts);
        end
        data_out.ds{k}{1,idim}=coords_new;
        dim_list_out=[dim_list_out,idim];
    end
    data_out.sas{k}=data_in.sas{k};
    %sets is taken from input, other than pipeline and dimensions present   
    data_out.sets{k}=data_in.sets{k};
    if isfield(xforms,'pipeline')
        data_out.sets{k}.pipeline=xforms.pipeline;
    else
        data_out.sets{k}.pipeline=struct();
    end
    data_out.sets{k}.pipeline.opts.opts_xform.transforms=xforms.ts;
    data_out.sets{k}.pipeline=filldefault(data_out.sets{k}.pipeline,'sets',data_in.sets{k});
    data_out.sets{k}.pipeline=filldefault(data_out.sets{k}.pipeline,'type','xform');
    data_out.sets{k}.dim_list=dim_list_out;
end
if ~isempty(missing_params)
    missing_params=unique(missing_params,'rows');
    miss_string=[];
    for imiss=1:size(missing_params,1)
        miss_string=cat(2,miss_string,' ',missing_params(imiss,:));
    end
    wmsg=sprintf('transform specification is missing parameters: %s',deblank(miss_string));
    aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_xform.if_warn));
end
aux_out.opts_check=aux.opts_check;
aux_out.opts_xform=aux.opts_xform;
return
end

`rs_xform_specify(data_in, aux)`

[xforms,aux_out]=rs_xform_specify(data_in,aux) creates a transformation structure in the affine model class based the coordinates in a dataset structure

A transformation structure is a cell array of geometric transformations. The transformation structures created by this module are combinations of linear transformations and translations. These transformations can be chosen to center the coordinates in data_in (i.e., move the centroid to the origin), to move the coordinates of a specific stimulus to the origin, and/or or rotate the coordinates into their principal components. To fit a geometric model to a transformation between coordinate sets, see rs_geofit

Input arguments:

data_in (struct) –
dataset structure to use for specifying the transformations, with fields
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record
aux (struct) –
auxiliary options, may be omitted, with fields
- opts_xform (struct) – specification of the transformation, with fields
  - mode (char) – One of 'none', 'translate','offset_pca','translate_then_pca'; default is 'none'
    - 'none': the identity transformation
    - 'translate': a point specified by 'centering_specifier' (see below) is translated to the origin
    - 'offset_pca': the data are rotated into the principal components around the point specified by 'centering_specifier' (see below). The first coordinate explains the most variance around that point, the second coordinate explains the next-most-variance, etc. The point specified by the 'centering_specifier' is not moved.
    - 'translate_then_pca': the point specified by 'centering_specifier') is translated to the origin, and then standard pca is done.
  - source (char) – 'global','local', or an integer in [1:n], where n is the number of records, equal to length(data_in.ds); default is 'global'
    - 'global' (default): the centering specifier is determined from the mean across records; pca is computed across all records; the transformations specified for all records are identical
    - 'local': the centering specifier and pca is computed separately for each record; tranformations for each record typically differ
    - an integer: the specified record is used for the centering specifier and pca; transformations foe all records are identical
  - centering_specifier (char) – 'none','centroid','index','typename','value'; default is 'none'
    - 'none': no centering is done
    - 'centroid': centroid is the centering point
    - 'typename': use the stimulus corresponding to aux.opts_xform.centering_typename in data_in.sas{k}.typenames for centering
    - 'index': use the value in aux.opts_xform.centering_index, as the index number of the stimulus whose coordinates are to be used for centering. Caution: the index refers to the position of the stimulus in data_in.ds{k}, and stimulus order may differ across datasets. To avoid mis-referencing stimuli, specify by typename.
    - 'value': use the coordinates in aux.opts_xform.,centering_value for centering
  - centering_typename (char) – the label of the stimulus in data_in.sas{k} to be used for centering
  - centering_index (int) – the index in data_in.ds{k} to be used for centering
  - centering_value (float 1-D array) – the coordinates to be used for centering; for coordinate sets of dimension k, only the first k are used aux.opts_xform.centering_[index|typename|value]: see above in aux.opts_xform.centering_specifier
  - if_warn (int) – 1 (default) to show warnings
- opts_check (struct) – options for consistency checking, with field
  - if_warn (int) – 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1

Output arguments:

xforms ( struct ) –
the transformations, with fields
- ts (cell array) – ts{k}{idim} is the transformation to be applied to the coordinates of dimension idim in record k. See note below regarding transformations
- class (string) – model class of the transformation; this is always 'affine'. See transformation structures
- pipeline (structure) – a structure that indicates how the transformation is specified, and can serve as the 'pipeline' field of a set metadata structure when the transformations are applied
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- opts_xform (struct) – aux.opts_xform, with defaults filled in
- opts_check (struct) – aux.opts_check, with defaults filled in

Note regarding transformations

The transformations specified by rs_xform_specify are combinations of translations and linear transformations, and belong to the model class 'affinedescribed intransformation structures`. With ts=xforms.ts{k}{idim}, and each input a row vector, the transformation is [output]=ts.b[input]ts.T+ts.c, where ts.b is a scalar, ts.T is an array of size [idim,idim], [input] is an array of size [nstims,idim], and c is a row vector of length idim.
This structure is identical to the output of Matlab's procrustes.m, [??how to hyperlink], except that the translation ts.c in procrustes.m is an array of identical rows; here it is just one row.
This structure is also identical to that of aux_out.knit_stats.ts{idim}{k}, where the roles of ts.b, ts.T, and ts.c are replaced by ts.scaling, ts.orthog, and ts.translation. ts_specify{k}{idim}=procrustes_compat(ts_knit{idim}{k}) can be used to convert the transformations produced by ts_knit_coordsets to the transformations produced by rs_xform_specify.

See also: rs_aux_customize, rs_check_coordsets, PSG_PCAOFFSET, rs_xform_specify_test, rs_xform_apply.

Source code in src/rs_xform_specify.m

function [xforms,aux_out]=rs_xform_specify(data_in,aux)
% [xforms,aux_out]=rs_xform_specify(data_in,aux)
% creates a `transformation structure` in the affine model class based the coordinates in a `dataset structure`
%
% A `transformation structure` is a cell array of geometric transformations.
% The `transformation structures` created by this module are combinations of linear transformations and translations.
% These transformations can be chosen to center the coordinates in data_in
% (i.e., move the centroid to the origin), to move the coordinates of a
% specific stimulus to the origin, and/or or rotate the coordinates into their principal components.
% To fit a geometric model to a transformation between coordinate sets, see `rs_geofit`
% 
% Args:
%   data_in (struct): `dataset structure` to use for specifying the transformations, with fields
%
%     - ds (cell array): `coordinate structure`, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
%     - sas (cell array): `stimulus metadata structure`, sas{k} is the stimulus metadata for the kth record
%     - sets (cell array): `set metadata structure`, sets{k} is the response metadata for the kth record
%
%   aux (struct): auxiliary options, may be omitted, with fields
%
%     - opts_xform (struct): specification of the transformation, with fields
%
%         - mode (char): One of 'none', 'translate','offset_pca','translate_then_pca'; default is 'none'
%
%             - 'none': the identity transformation
%             - 'translate': a point specified by 'centering_specifier' (see below) is translated to the origin
%             - 'offset_pca': the data are rotated into the principal components around the point specified by 'centering_specifier' (see below). 
%             The first coordinate explains the most variance around that point, the second coordinate explains the next-most-variance, etc.
%             The point specified by the 'centering_specifier' is not moved.
%             - 'translate_then_pca': the point specified by 'centering_specifier') is translated to the origin, and then standard pca is done.
%
%         - source (char): 'global','local', or an integer in [1:n], where n is the number of records, equal to length(data_in.ds); default is 'global'
%
%             - 'global' (default): the centering specifier is determined from the mean across records; pca is computed across all records;
%             the transformations specified for all records are identical
%             - 'local': the centering specifier and pca is computed separately for each record; tranformations for each record typically differ
%             - an integer: the specified record is used for the centering specifier and pca; transformations foe all records are identical
%
%         - centering_specifier (char): 'none','centroid','index','typename','value'; default is 'none'
%
%             - 'none': no centering is done
%             - 'centroid': centroid is the centering point
%             - 'typename': use the stimulus corresponding to aux.opts_xform.centering_typename in data_in.sas{k}.typenames for centering
%             - 'index': use the value in aux.opts_xform.centering_index, as the
%             index number of the stimulus whose coordinates are to be used for centering.
%             Caution: the index refers to the position of the stimulus in data_in.ds{k}, and stimulus order may differ across datasets. 
%             To avoid mis-referencing stimuli, specify by typename.
%             - 'value': use the coordinates in aux.opts_xform.,centering_value for centering
%
%         - centering_typename (char): the label of the stimulus in data_in.sas{k} to be used for centering
%         - centering_index (int): the index in data_in.ds{k} to be used for centering
%         - centering_value (float 1-D array): the coordinates to be used for centering; for coordinate sets of dimension k, only the first k are used
%         aux.opts_xform.centering_[index|typename|value]: see above in aux.opts_xform.centering_specifier
%         - if_warn (int): 1 (default) to show warnings
%
%     - opts_check (struct): options for consistency checking, with field
%
%          - if_warn (int): 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
% 
% Returns:
%   xforms (struct):  the transformations, with fields
%
%      - ts (cell array): ts{k}{idim} is the transformation to be applied to the coordinates of dimension idim in record k. See note below regarding transformations
%      - class (string): model class of the transformation; this is always 'affine'. See `transformation structures` 
%      - pipeline (structure): a structure that indicates how the transformation is specified, and can serve as the 'pipeline' field of a `set metadata structure` when the transformations are applied
%
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - opts_xform (struct): aux.opts_xform, with defaults filled in
%     - opts_check (struct): aux.opts_check, with defaults filled in
%
% Note: Note regarding transformations
%     - The transformations specified by `rs_xform_specify` are combinations of translations
%     and linear transformations, and belong to the model class 'affine` described in `transformation structures`.  With ts=xforms.ts{k}{idim}, and each
%     input a row vector, the transformation is
%     [output]=ts.b*[input]*ts.T+ts.c, where ts.b is a scalar, ts.T is an
%     array of size [idim,idim], [input] is an array of size [nstims,idim], and c is a row vector of length idim.
%     - This structure is identical to the output of Matlab's procrustes.m,
%     [??how to hyperlink], except that the translation ts.c in procrustes.m
%     is an array of identical rows; here it is just one row.
%     - This structure is also identical to that of
%     aux_out.knit_stats.ts{idim}{k}, where the roles of ts.b, ts.T, and ts.c are
%     replaced by ts.scaling, ts.orthog, and ts.translation.
%     ts_specify{k}{idim}=procrustes_compat(ts_knit{idim}{k}) can be used
%     to convert the transformations produced by `ts_knit_coordsets` to the transformations produced by `rs_xform_specify`.
%
%  See also: RS_AUX_CUSTOMIZE, RS_CHECK_COORDSETS, PSG_PCAOFFSET, RS_XFORM_SPECIFY_TEST, RS_XFORM_APPLY.
%
if (nargin<=1)
    aux=struct;
end
%
aux=filldefault(aux,'opts_xform',struct); 
%
aux.opts_xform=filldefault(aux.opts_xform,'mode','none'); %'none', 'translate', 'offset_pca', 'translate_then_pca'; 
aux.opts_xform=filldefault(aux.opts_xform,'source','global'); %options: global: calculation based on mean across datasets, 'local', use each dataset's value, or a number (specify the dataset to use)
aux.opts_xform=filldefault(aux.opts_xform,'centering_specifier','none'); %'none','centroid','index','typename','value'
aux.opts_xform=filldefault(aux.opts_xform,'centering_typename','rand'); %specify the typename to move to the origin
aux.opts_xform=filldefault(aux.opts_xform,'centering_index',1); %specify the stimulus index to move to the origin
aux.opts_xform=filldefault(aux.opts_xform,'centering_value',[]); %specify coordinate value to move to the origin
aux.opts_xform=filldefault(aux.opts_xform,'if_warn',1); %show warnings
%
aux=filldefault(aux,'opts_check',struct);
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
aux=rs_aux_customize(aux,'rs_xform_specify');
%
xforms=struct;
xforms.ts=cell(0);
xforms.pipeline=struct;
%
aux_out=struct;
aux_out.opts_check=aux.opts_check;
%
%check consistency and get available stimuli, dimensions, typenames
%
check=rs_check_coordsets(data_in,aux.opts_check);
%
aux_out.warnings=check.warnings;
aux_out.warn_bad=check.warn_bad;
nsets=check.nsets;
nstims_each=check.nstims_each;
dim_list_each=check.dim_list_each;
dim_list_union=check.dim_list_union;
dim_list_inter=check.dim_list_inter;
typenames_each=check.typenames_each;
typenames_union=check.typenames_union;
typenames_inter=check.typenames_inter;
%
%validate input parameters for consistency, etc.
%
if_ok_centering=1;
if_ok_mode=1;
x=aux.opts_xform; %for convenience
%
switch x.centering_specifier
    case {'none','centroid'} 
        x.centering_index=[];
        x.centering_value=[];
        x.centering_typename=[];
    case 'index'
        if ((x.centering_index~=round(x.centering_index)) | (x.centering_index>min(nstims_each)) | (x.centering_index<=0))
            wmsg=sprintf('centering index (%8.3f) not valid (non-integer or out of range); no centering applied',x.centering_index);
            aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_xform.if_warn));
            if_ok_centering=0;
        else
            idx=x.centering_index;
        end
        x.centering_value=[];
        x.centering_typename=[];
    case 'value'
        if length(x.centering_value)<max(dim_list_union)
            wmsg=sprintf('centering value vector length (%3.0f) less than max dimension needed (%3.0f); no centering applied',length(x.centering_value),max(dim_list_union));
            aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_xform.if_warn));
            if_ok_centering=0;
        else
            x.centering_value=x.centering_value(:)';
        end
        x.centering_index=[];
        x.centering_typename=[];
    case 'typename'
         idx_check=strmatch(x.centering_typename,typenames_inter,'exact');
         if length(idx_check)~=1
            wmsg=sprintf('centering typename (%s) not found or not unique in all datasets; no centering applied',x.centering_typename);
            aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_xform.if_warn));
            if_ok_centering=0;
         end
        x.centering_index=[];
        x.centering_value=[];
    otherwise
        wmsg=sprintf('centering specifier (%s) not recognized; no centering applied',x.centering_specifier);
        if_ok_centering=0;
        aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_xform.if_warn));
        x.centering_index=[];
        x.centering_value=[];
        x.centering_typename=[];
end
switch x.source
    case {'global','local'}
    otherwise
        if isnumeric(x.source)
            if ((x.source~=round(x.source)) | (x.source>nsets) | (x.source<=0))
               wmsg=sprintf('source (%8.3f) not valid (non-integer or out of range); no centering applied',x.source);
               aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_xform.if_warn));
               if_ok_centering=0;
            end
        else
           wmsg=sprintf('centering source (%s) not recognized; no centering applied',x.source);
           aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_xform.if_warn));
           if_ok_centering=0;
        end
end
switch x.mode
    case {'none','translate','offset_pca','translate_then_pca'}
    otherwise
        wmsg=sprintf('mode  (%s) not recognized; no transformation applied',x.mode);
        aux_out=rs_warning(wmsg,0,setfield(aux_out,'if_warn',aux.opts_xform.if_warn));
        if_ok_mode=0;
end
if (if_ok_centering==0)
    x.centering_specifier='none';
end
if (if_ok_mode==0)
    x.mode='none';
end
aux_out.opts_xform=x;
%
xforms.pipeline.type='xform';
xforms.pipeline.opts.opts_xform=aux.opts_xform;
xforms.pipeline.sets_combined=cell(1,nsets);
for iset=1:nsets
    xforms.pipeline.sets_combined{iset}=data_in.sets{iset};
end
%
if aux_out.warn_bad==0
    %process
    xforms.ts=cell(nsets,1);
    nstims=min(nstims_each);
    for iset=1:nsets
        xforms.ts{iset}=cell(1,max(dim_list_each{iset}));
        for dim_ptr=1:length(dim_list_each{iset});
            idim=dim_list_each{iset}(dim_ptr);
            %
            ts=struct;
            ts.b=1;
            ts.T=eye(idim);
            ts.c=zeros(1,idim);
            if ~strcmp(x.mode,'none')
                switch x.source
                    case 'local'
                        source=iset;
                        have_data=ismember(idim,dim_list_each{iset});
                    case 'global'
                        source=[1:nsets];
                        have_data=ismember(idim,dim_list_inter);
                    otherwise
                        source=x.source;
                        have_data=1;
                end
                if have_data
                    %determine centering
                    coords_each=zeros(nstims,idim,length(source));
                    for is=1:length(source)
                        coords_each(:,:,is)=data_in.ds{source(is)}{idim};
                    end
                    coords=mean(coords_each,3,'omitnan');
                    switch x.centering_specifier % none','centroid','index','typename','value'
                        case 'none'
                            cvec=zeros(1,idim);
                        case 'centroid'
                            cvec=mean(coords,1,'omitnan');
                        case 'index'
                            cvec=coords(idx,:);
                        case 'typename'
                            cvec_each=zeros(1,idim,length(source));
                            for is=1:length(source)
                                cvec_each(1,:,is)=coords_each(strmatch(x.centering_typename,typenames_each{source(is)},'exact'),:,is);
                            end
                            cvec=mean(cvec_each,3,'omitnan');
                        case 'value'
                            cvec=x.centering_value(1:idim);
                    end
                    %handle rotation
                    switch x.mode % 'none', 'translate', 'offset_pca', 'translate_then_pca';
                        case 'none'
                        case 'translate'
                            ts.c=-cvec;
                        case {'offset_pca','translate_then_pca'}
                            if (iset==1) | ~strcmp(x.source,'global')
                                %cvec+(coords-cvec)*qv is the reconstruction in the PC space.
                                [recon_pcaxes,recon_coords,var_ex,var_tot,coord_maxdiff,opts_offset_pca]=psg_pcaoffset(coords,cvec);
                                qv=opts_offset_pca.qv;
                                ts.T=qv;
                                if strcmp(x.mode,'offset_pca')
                                    ts.c=-cvec*qv+cvec;
                                else
                                    ts.c=-cvec*qv;
                                end
                            else
                                ts=xforms.ts{1}{idim}; %no need to redo PCA
                            end
                    end
                else
                    ts=[]; %better than struct(), which is not empty
                end
            end %mode
            xforms.ts{iset}{idim}=ts;
            xforms.class='affine';
        end %dim_ptr
    end %each set
else
    disp('cannot proceed');
    disp(aux_out.warnings);
end
return
end

`rs_aux_defaults_define_dist`

rs_aux_defaults_define sets the defaults for auxiliary input parameters.

Run this file in a clean workspace, and save the resulting workspace in rs_aux_defaults.mat.

The resulting file rs_aux_defaults.mat is read by rs_aux_customize at run-time, to set defaults for auxiliary input parameters.

This file, rs_aux_defaults_define.m (or rs_aux_defaults_define_dist.m), is intended for distribution and customization. If inadvertently over-written, a spare copy is in rs_aux_defaults_define_dist.m, and the mat-file it creates is saved in rs_aux_defaults_dist.mat.

Search for '[' to find typical fields to be customized.

`rs_aux_defaults_define`

rs_aux_defaults_define sets the defaults for auxiliary input parameters.

Run this file in a clean workspace, and save the resulting workspace in rs_aux_defaults.mat.

The resulting file rs_aux_defaults.mat is read by rs_aux_customize at run-time, to set defaults for auxiliary input parameters.

This file, rs_aux_defaults_define.m (or rs_aux_defaults_define_dist.m), is intended for distribution and customization. If inadvertently over-written, a spare copy is in rs_aux_defaults_define_dist.m, and the mat-file it creates is saved in rs_aux_defaults_dist.mat.

Search for '[' to find typical fields to be customized.

`rs_aux_defaults_define_pvt`

rs_aux_defaults_define_pvt sets the defaults for auxiliary input parameters.

This version of the file is intended for local use in the Victor lab and is provided as an example of customization; see rs_aux_defaults_define for distribution version.

Run this file in a clean workspace, choose whether to take the for binary texture data or calcium imaging data, and, if these defaults are to be used, save the resulting workspace in rs_aux_defaults.mat. The resulting file rs_aux_defaults.mat is read by rs_aux_customize at run-time, to set defaults for auxiliary input parameters.

Results of choosing the binary texture option are saved in rs_aux_defaults_btc.mat. Results of choosing the calcium imaging option are saved in rs_aux-defaults_hlid.mat.

`/src/utils`

`compare_signflip(x, y, dimflip)`

[ifdif,maxdiff,maxdiff_noflip,diffs,diffs_noflip,signs]=compare_signflip(x,y,dimflip) compares two numeric arrays, allowing for sign flips

This is a utility used to verify computations against benchmarks, recognizing that PCA implementations may differ in sign assignemnts.

Input arguments:

x (array or cell array) –

one array to be compared
y (array or cell array) –

second array to be compared
dimflip (int) –

dimension that can be flipped, defaults to 1

Output arguments:

ifdif ( char ) –

empty if x and y are identical or identical after sign flips along dimension dimflip; otherwise, 'different types', 'different shapes' or 'different values', depending on first difference encountered
maxdiff ( float ) –

maximum absolute value of difference between x and y, after sign flips along dimflip to minimize differences
maxdiff_noflip ( float ) –

maximum absolute value of difference between x and y, without sign flips
diffs ( float array ) –

maximum absolute value of difference between x and y along the dimension dimflip, after sign flips to minimize differences
diffs_noflip ( float array ) –

maximum absolute value of difference between x and y along the dimension dimflip, without sign flips
signs ( int ) –

+1 or -1, indicating whether the difference between x and y is minimized by a flip; 0 if x and y are both 0

Array outputs

The arrays diffs, diffs_noflip, and signs are empty if x and y differ in type or shape, and therefore cannort be compared numerically.

If x and y can be compared numerically, the arrays diffs, diffs_noflip, and signs all have same shape as x and y except on the dimension dimflip, where they have length 1

Source code in src/utils/compare_signflip.m

function [ifdif,maxdiff,maxdiff_noflip,diffs,diffs_noflip,signs]=compare_signflip(x,y,dimflip)
% [ifdif,maxdiff,maxdiff_noflip,diffs,diffs_noflip,signs]=compare_signflip(x,y,dimflip) compares two numeric arrays, allowing for sign flips
% 
% This is a utility used to verify computations against benchmarks, recognizing that PCA implementations may differ in sign assignemnts.
% 
% Args:
%   x (array or cell array): one array to be compared
%
%   y (array or cell array): second array to be compared
%
%   dimflip (int): dimension that can be flipped, defaults to 1
%
% Returns:
%   ifdif (char): empty if x and y are identical or identical after sign flips along dimension dimflip; otherwise, 'different types', 'different shapes' or 'different values', depending on first difference encountered
%
%   maxdiff (float): maximum absolute value of difference between x and y, after sign flips along dimflip to minimize differences
%
%   maxdiff_noflip (float): maximum absolute value of difference between x and y, without sign flips
% 
%   diffs (float array): maximum absolute value of difference between x and y along the dimension dimflip, after sign flips to minimize differences
%
%   diffs_noflip (float array): maximum absolute value of difference between x and y along the dimension dimflip, without sign flips
%
%   signs (int): +1 or -1, indicating whether the difference between x and y is minimized by a flip; 0 if x and y are both 0
%
% Note: Array outputs
%   The arrays diffs, diffs_noflip, and signs are empty if x and y differ in type or shape, and therefore cannort be compared numerically.
%
%   If x and y can be compared numerically, the arrays diffs, diffs_noflip, and signs all have same shape as x and y except on the dimension dimflip, where they have length 1
%
if (nargin<=2)
    dimflip=1;
end
ifcx=iscell(x);
ifcy=iscell(y);
ndx=ndims(x);
ndy=ndims(y);
sx=size(x);
sy=size(y);
%
ifdif=[];
diffs=[];
diffs_noflip=[];
maxdiff=[];
maxdiff_noflip=[];
signs=[];
%
if (ifcx~=ifcy)
    ifdif='different types';
else
    if ndx~=ndy
        ifdif='different shapes';
    elseif any(sx~=sy)
        ifdif='different shapes';
    else % size and shape match
        if ifcx==1 %cell, so recurse
            xr=x(:);
            yr=y(:);
            nrec=length(xr);
            diffs_rec=cell(1,nrec);
            diffs_noflip_rec=cell(1,nrec);
            signs_rec=cell(1,nrec);
            maxdiff=0;
            maxdiff_noflip=0;
            for k=1:length(xr)
                [ifdif_rec{k},maxdiff_rec,maxdiff_noflip_rec,diffs_rec{k},diffs_noflip_rec{k},signs_rec{k}]=compare_signflip(xr{k},yr{k},dimflip);
                if ~isempty(ifdif_rec{k})
                    if isempty(ifdif)
                        ifdif=ifdif_rec{k};
                    end
                end
                if ~isempty(maxdiff_rec)
                    maxdiff=max(maxdiff,maxdiff_rec);
                    maxdiff_noflip=max(maxdiff_noflip,maxdiff_noflip_rec);
                end
            end
            diffs=reshape(diffs_rec,sx);
            diffs_noflip=reshape(diffs_noflip_rec,sx);
            signs=reshape(signs_rec,sx);
        else %do comparisons
            diffs_noflip=max(abs(x-y),[],dimflip);
            diffs_allflip=max(abs(x+y),[],dimflip);
            diffs=min(cat(dimflip,diffs_noflip,diffs_allflip),[],dimflip);
            maxdiff=max(diffs(:));
            maxdiff_noflip=max(diffs_noflip(:));
            signs=sign(diffs_allflip-diffs_noflip);
            if maxdiff>0
                ifdif='different values';
            end
        end
    end
return
end

`compstruct(parent1, x1, parent2, x2, varargin)`

ifdif=compstruct(parent1,x1,parent2,x2,varargin) compares two structures

This function is adapted from a file "compare.m" downloaded from the MATLAB file exchange on 6/4/04, from Nicholas Gigis, gigisn@netscape.net. Original documentation is retained at the beginning of the code.

Input arguments:

parent1 (char) –

name of first structure to be compared
x1 (struct) –

first structure to be compared
parent2 (char) –

name of second structure to be compared
x2 (struct) –

second structure to be compared

Output arguments:

ifdif ( char ) –

text string of differences, empty if structures are identical

Source code in src/utils/compstruct.m

function ifdif=compstruct(parent1,x1,parent2,x2,varargin)
% ifdif=compstruct(parent1,x1,parent2,x2,varargin) compares two structures
%
% This function is adapted from a file "compare.m" downloaded from the MATLAB file exchange on 6/4/04, from Nicholas Gigis, gigisn@netscape.net.
% Original documentation is retained at the beginning of the code.
%
% Args:
%   parent1 (char): name of first structure to be compared
%   
%   x1 (struct): first structure to be compared
%
%   parent2 (char): name of second structure to be compared
%   
%   x2 (struct): second structure to be compared
%
% Returns:
%   ifdif (char): text string of differences, empty if structures are identical
%

%
%  Modifications by JV:
%  Modified so that the text is returned as an output; empty if structures are identical
%  13Apr21: edit made to fix bug in comparenum when inputs have different dimensions
%  26Dec25: edit made to avoid errors if grpahics objects are being compared
%  06Feb26: edit made to avoid errors if cell arrays are empty and have different dimensions
%
% Original documentation from Nicholas Gigis, gigisn@netscape.net, follows.
%
% Compare - This function compares two objects and outputs the
%           locations of differences between the two.  This function
%           handles nested structures, nested subarrays, arrays, strings,
%           and graphic objects.  In the case of objects, if the handles
%           are passed to x1 and x2, the properties are obtained with the
%           get function and the properties are compared.
%
%
% parent1  = name of first structure
% x1       = first structure
% parent2  = name of second structure
% x2       = second structure
% OPTS     = print and tolerance options
%            tol      = tolerance for comparison between floats
%            filename = if you wish to print the differences to a file, enter a
%                       filename for it to be save as.  Must be in single quotes.
%SPAWAR System Center Charleston
%Code 712 DF Calibraton Group
%
%  Ver: 1.0  5/19/03
%
ifdif=[];
if nargin<4;
    error('Incorrect usage of Compare');
elseif nargin==4;
    tol=1e-5;
    fid=1;
elseif nargin==5;
    if strcmp(class(varargin{1}),'char')
        fid=fopen(varargin{1},'w');
        tol=1e-5;
    elseif strcmp(class(varargin{1}),'double');
        fid=1;
        tol=varargin{1};
    else;
        error('Incorrect usage of Compare');
    end;
elseif nargin==6;
    if strcmp(class(varargin{1}),'char') & strcmp(class(varargin{2}),'double');
        fid=fopen(varargin{1},'w');
        tol=varargin{2};
    elseif strcmp(class(varargin{2}),'char') & strcmp(class(varargin{1}),'double');
        fid=fopen(varargin{2},'w');
        tol=varargin{1};
    else;
        error('Incorrect usage of Compare');
    end;
elseif nargin>6;
    error('Incorrect usage of Compare');
end;

if ishandle(x1) & ishandle(x2);
    x1=get(x1);
    x2=get(x2);
end;

if ~isequal(x1,x2) & strcmp(class(x1),class(x2));   %if x1 is not equal to x2 and they are the same class
    if isstruct(x1);                                %if x1 a structure send to function comparestruct
        ifdif=comparestruct(parent1,x1,parent2,x2,tol,fid);
    elseif iscell(x1);                              %if x1 a cell send to function comparecell
        ifdif=comparecell(parent1,x1,parent2,x2,tol,fid);
    else;                                           %if x1 is any of the remaining classes, send to function comarenum
        ifdif=comparenum(parent1,x1,parent2,x2,tol,fid);
    end;
elseif ~strcmp(class(x1),class(x2));                %if x1 and x2 are not the same class
    ifdif=['Classes are different: ' parent1 ' is a ' class(x1) ' , and ' parent2 ' is a ' class(x2)];
end;
if fid~=1;
   fprintf(fid,[ifdif '\n']);
    fclose(fid);        
end;




function ifdif=comparenum(parent1,x1,parent2,x2,tol,fid);     %function to compare numerics, strings, logicals, & chars

`extorthb(cv)`

[basis,onb]=extorthb(cv) extends a column vector to an orthogonal basis

The extension is attempted in a numerically "good" way, not by simply doing a Gram-Schmidt procedure

Input arguments:

cv (float 1-D array) –

a column vector

Output arguments:

basis ( float 2-D array ) –

a square matrix, size(basis)=[length(cv) length(cv)]; basis(:,1)=cv, basis'*basis is diagonal
onb ( float 2-D array ) –

corresonding orthonormal basis, vectors in columns; onb'*onb=1

`extorthb_gen(v)`

[basis,onb]=extorthb_gen(v) extends a set of column vectors to an orthogonal basis

The extension is attempted in a numerically "good" way, not by simply doing a Gram-Schmidt procedure

Input arguments:

v (float 2-D array) –

a set of column vectors, assumed to be orthonormal.

Output arguments:

basis ( float 2-D array ) –
basis a square matrix, size(basis)=[size(v,1) size(v,1)], basis'basis and basisbasis' is the identity
- If v is orthonormal, then basis(:,1:size(v,2))=v
- If v is not orthonormal, then v is in the span of the first size(v,2) columns of basis, with v(:,1) proportional to basis(:,1), v(:,k) in the span of basis(:,1:k). Coefficients are in in basis'*v, which is upper-triangular
onb ( float 2-D array ) –

same as basis, for backwards compatibility

`fibspiral(n, d, opts)`

x=fibspiral(n,d) produces a sampling of points on the surface of a d-dimensional hypersphere

The method is inspired by this Math Stack Exchange post

Input arguments:

n (int) –

number of points
d (int) –
dimension. Should work for any number of dimensions >=1
- d=1 yields samples that alternate between +1 and -1
- d=2 yields smaples equally spaced around a circle
- d>=3 yeilds nontrivial spirals
opts (struct) –
options structure, with fields
- nangs_min (int) – minimal sampling for the axial angle; default is 1024
- nangs_mult (int) – minimal factor for the number of points in sampling the axial angle; default is 16
- mults (float 1-D array) – list of incommensurate numbes of length at least d-1; if shorter or not provided, square roots of lowest primes are used

Output arguments:

x ( int ) –

array of size[n d] of the sampled points, magnitude 1
opts_used ( struct ) –

options used

`filldefault(opts, names, val)`

optsused=filldefault(opts,names,val) fills an option structure with default values

Input arguments:

opts (struct) –

an option structure, possibly with some fields missing
names (char or char 2-D array) –

field name or names to be supplied with default values
val (int, float, char, cell, or struct) –

the value to use as default, for each field in 'names' that is not present in 'opts'

Output arguments:

optsused ( struct ) –

opts, with missing fields filled in

Source code in src/utils/filldefault.m

function optsused=filldefault(opts,names,val)
% optsused=filldefault(opts,names,val) fills an option structure with default values
%
% Args:
%   opts (struct): an option structure, possibly with some fields missing
%
%   names (char or char 2-D array): field name or names to be supplied with default values
%
%   val (int, float, char, cell, or struct): the value to use as default, for each field in 'names' that is not present in 'opts'
%
% Returns:
%   optsused (struct): opts, with missing fields filled in
% 
optsused=opts;
for iname=1:size(names,1)
    fieldname=deblank(names(iname,:));
    if (~isfield(opts,fieldname))
%        optsused=setfield(optsused,fieldname,val);
        optsused.(fieldname)=val;
    end
end
return

`getinp(prompt, type, limits, default)`

vals=getinp(prompt,type,limits,default) gets an input value or values from the console and supplies a default

Input arguments:

prompt (char) –

a prompt string, to appear after "Enter"
type (char) –

'd','f','s' for integer, float, or string input; input value is required to be an integer if 'd' is used
limits (int 1-D array or float 1-D arrray) –

limits(1) and limits(2) are low and high limits for input; ignored and may be empty if 'type'='s'
default (int 1-D array or float 1-D array) –

default value(s) used if console response is empty

Output arguments:

vals ( int 1-D array, float 1-D array, or char ) –

value(s) entered at console, or default if no value is entered

Source code in src/utils/getinp.m

function vals=getinp(prompt,type,limits,default)
% vals=getinp(prompt,type,limits,default) gets an input value or values from the console and supplies a default
%
% Args: 
%   prompt (char): a prompt string, to appear after "Enter"
%
%   type (char): 'd','f','s' for integer, float, or string input; input value is required to be an integer if 'd' is used
%   
%   limits (int 1-D array or float 1-D arrray): limits(1) and limits(2) are low and high limits for input; ignored and may be empty if 'type'='s'
% 
%   default (int 1-D array or float 1-D array): default value(s) used if console response is empty
%
% Returns:
%   vals (int 1-D array, float 1-D array, or char): value(s) entered at console, or default if no value is entered
% 
if (nargin<=3) default=[]; end
vals=[];
while (isempty(vals))
   switch type
   	case {'f','d'}
      	if (length(default)>1)
         	disp(default)
         	vals=input(sprintf(cat(2,'Enter ',prompt,' (range: %',type,' to %',type,'):'),...
            	limits([1 2])));
      	elseif (length(default)==1)
         	vals=input(sprintf(cat(2,'Enter ',prompt,' (range: %',type,' to %',type,', default= %',type,'):'),...
            	limits([1 2]),default));
      	else
        		vals=input(sprintf(cat(2,'Enter ',prompt,' (range: %',type,' to %',type,'):'),...
            	limits([1 2])));
      	end
         if (isempty(vals));vals=double(default);end %double just in case default is logical (ML7)
         if (min(vals)<limits(1)) | (max(vals)>limits(2));disp('Out of range.');vals=[];end
         if type=='d'
            if (max(abs(vals-floor(vals))))>0; disp('Must be integer.');vals=[];end
         end

   	case 's'
      	if (length(default)>=1)
            vals=input(sprintf(cat(2,'Enter ',prompt,' (default= ',default,'):')),type);
      	else
            vals=input(sprintf(cat(2,'Enter ',prompt,':')),type);
      	end
      	if (isempty(vals));vals=default;end
   end
end

`getsubfield(s, fns)`

x=getsubfield(s,fns) gets a (possibly deeply) nested subfield

Input arguments:

s (struct) –

a structure
fns (cell array) –

field names, fns={n₁, n₂,..., n_k}

Output arguments:

x ( int, float, cell, or struct ) –

the nested subfield s.(n₁).(n₂). ... .(n_k)

Notes

If any of the subfields are not present, x=[].

Subfield arrays not supported.

Uses recursion:

k>=2: getsubfield(s,{n₁, n₂,..., n_k})=getsubfield(s.(n₁),{n₂,..., n_k})
k=1: getsubfield(s,{n₁})=s.(n₁)

`grmscmdt(a)`

[gs,gsn]=grmscmdt(a) applies the Gram-Schmidt procedure to a set of column vectors

Input arguments:

a (float 2-D array) –

a set of column vectors

Output arguments:

gs ( float 2-D array ) –

orthogonal column vectors; span of first k columns match those of a
gsn ( float 2-D array ) –

orthonormal column vectors; span of first k columns match those of a

Notes

If some columns of a are linearly dependent, then gs and gsn will have some columns that are zero.

This works if a is complex. gs'*gs will be diagonal, and gsn'*gsn will be the identity.

`hsphere_sample(d, opts)`

[pts,opts_used]=hsphere_sample(d,opts) samples the surface of a hypersphere with a requested number of points

Input arguments:

d (int) –

number of dimensions (2=circle, 3=ordinary sphere)
opts (struct) –
options structure, with fields
- if_hemisphere (int) – 0 to sample whole hypersphere, 1 to sample only the hemi-hypersphere in which first coordinate is >0; default is 0
- method (char) – sampling method to use, with the following options; default is 'random'
  - 'axes': one point on each axis, yields 2*d/(1+opts.if_hemisphere) points
  - 'orthants': one point in each orthant, yields 2^d/(1+opts.if_hemisphere) points
  - 'axes_and_orthants': one point on each axis and in each orthant, combines 'axes' and 'orthants'
  - 'random': random
  - 'fibspiral': Fibonacci spiral
- nsamps (int) – number of points to sample, ignored if method='random' or 'fibspiral'; default is 2^d for 'random' and 4^d for 'fibspiral'

Output arguments:

pts ( float 2-D array ) –

array of size [nsamps d]; each row is a unit-length point on the hypersphere
opts_used ( struct ) –

options used

See also: filldefault, int2nary, fibspiral.

Source code in src/utils/hsphere_sample.m

function [pts,opts_used]=hsphere_sample(d,opts)
% [pts,opts_used]=hsphere_sample(d,opts) samples the surface of a hypersphere with a requested number of points
%
% Args:
%   d (int): number of dimensions (2=circle, 3=ordinary sphere)
%  
%   opts (struct): options structure, with fields
%
%      - if_hemisphere (int): 0 to sample whole hypersphere, 1 to sample only the hemi-hypersphere in which first coordinate is >0; default is 0
%
%      - method (char): sampling method to use, with the following options; default is 'random'
%
%          - 'axes': one point on each axis, yields 2\*d/(1+opts.if_hemisphere) points
%          - 'orthants': one point in each orthant, yields 2<sup>d</sup>/(1+opts.if_hemisphere) points
%          - 'axes_and_orthants': one point on each axis and in each orthant, combines 'axes' and 'orthants'
%          - 'random': random
%          - 'fibspiral': Fibonacci spiral
%
%      - nsamps (int): number of points to sample, ignored if method='random' or 'fibspiral'; default is 2<sup>d</sup> for 'random' and 4<sup>d</sup> for 'fibspiral' 
%
% Returns:
%   pts (float 2-D array): array of size [nsamps d]; each row is a unit-length point on the hypersphere
%
%   opts_used (struct): options used
%
% See also:  FILLDEFAULT, INT2NARY, FIBSPIRAL.
%
if (nargin<2)
    opts=struct;
end
opts=filldefault(opts,'method','random');
opts=filldefault(opts,'if_hemisphere',0);
switch opts.method
    case 'axes'
        nsamps=(2*d)/(1+opts.if_hemisphere);
    case 'orthants'
        nsamps=(2^d)/(1+opts.if_hemisphere);
    case 'axes_and_orthants'
        nsamps=(2*d+2^d)/(1+opts.if_hemisphere);
    case 'random'
        opts=filldefault(opts,'nsamps',2^d);
        nsamps=opts.nsamps;
    case 'fibspiral'
        opts=filldefault(opts,'nsamps',4^d);
        nsamps=opts.nsamps;
    otherwise
        nsamps=0;
        warning(sprintf('method %s not recognized',opts.method));
        pts=[];
end
opts_used=opts;
if strfind(opts.method,'orthants')
    pts_orthants=fliplr([(1-2*int2nary([0:2^(d-opts.if_hemisphere)-1]',2,d))/sqrt(d)]);
end
if strfind(opts.method,'axes')
    pts_axes=eye(d);
    if opts.if_hemisphere==0
        pts_axes=[pts_axes;-eye(d)];
    end
end
switch opts.method
    case 'axes'
        pts=pts_axes;
    case 'orthants'
        pts=pts_orthants;
    case 'axes_and_orthants'
        if d>1
            pts=[pts_axes;pts_orthants];
        else
            pts=pts_axes;
            nsamps=2/(1+opts.if_hemisphere);
        end
    case 'random'
        pts=randn(nsamps,d);
        pts=pts./repmat(sqrt(sum(pts.^2,2)),[1 d]);
        if opts.if_hemisphere
            pts(:,1)=abs(pts(:,1));
        end
    case 'fibspiral'
        [pts,ou]=fibspiral(nsamps,d,opts);
        if opts.if_hemisphere
            pts(:,1)=abs(pts(:,1));
        end
        fields=fieldnames(ou);
        for ifn=1:length(fields)
            opts_used=filldefault(opts_used,fields{ifn},ou.(fields{ifn}));
        end
end
opts_used.nsamps=nsamps;
return
end

`int2nary(ints, d, pmin)`

nary=int2nary(ints,d,pmin) converts one or more integers to n-ary vectors

The resulting vectors run along the first unused dimesion of ints, as integers base d. The least significant bit is in position 1, and zero-padding is used if needed. Generalized n-ary conversion is also done.

Input arguments:

ints (int or int array) –

the integers
d (int or int array) –

the base, default is 2 if omitted or empty. d may be a vector; see note below regarding generalized n-ary conversion
pmin (int) –

minimum length of the created dimension; if omitted, minimum possible value is used

Output arguments:

nary ( int array ) –

the converted array of n-ary numbers

Generalized n-ary conversion

If d is a vector, then d(1) is used for the first place, d(2) for the second, etc. and d(end) used if d does not have sufficient length.

In this case, ints(k)=nary(k,1)+d(1)*nary(k,2)+d(1)*d(2)*nary(k,3)+...

This reduces to the usual n-ary representation if d is a scalar

`isgraphc(connect, if_fast)`

[isgraph,msg,niters]=isgraphc(connect,if_fast) determines whether a matrix is consistent with a connected graph

This works for directed graphs, and can also be used to compute a graph's diameter.

Input arguments:

connect (int 2-D array) –

a square connectivity matrix: connect(i,j)=1 if there is a connection from node i to node j. For directed graphs, the matrix will not be symmetric.
if_fast (int) –
algorithm choice, default is 0
- 0: use sparse matrix methods rather than squaring; on return, niters+1 is the graph diameter
- 1: use successive squaring; on return, niters+1 is not necessarily the graph diameter

Output arguments:

isgraph ( int ) –

1 if graph is connected and symmetric, -1 if connected but directed, 0 if not connected or not a graph
msg ( int ) –

reason for isgraph~=1
niters ( int ) –

number of iterations used

`loglik_beta_discrete(ab, obs, opts)`

[loglik,opts_used]=loglik_beta_discrete(ab,obs,opts) determines the log likelihood (natural log) for a beta distribution (Dirichlet prior) with discrete part, based on a set of observations

Input arguments:

ab (int 1-D array) –

Dirichlet parameters a=ab(1), b=ab(2), or a=b=ab(1) if scalar; a and b must be >0
obs (int 2-D array) –

a vector of observations where obs(:,1) is successes and obs(:,2) is total tries
opts (struct) –
options, with fields
- hvec (float 1-D array) – vector of weights for point-mass probabilities, must be all >=0 and sum to <=1 (not checked for), defaults to [];
- qvec (float 1-D array) – vector of probabilities corresponding to the elements of hvec, must all be strictly >0 and < 1

Output arguments:

loglik ( float ) –

log likelihood (natural log)
opts_used ( struct ) –

option values used

See also: LOGLIK_BETA, BETALN, filldefault.

Source code in src/utils/loglik_beta_discrete.m

function [loglik,opts_used]=loglik_beta_discrete(ab,obs,opts)
% [loglik,opts_used]=loglik_beta_discrete(ab,obs,opts) determines the log likelihood (natural log) for a beta distribution (Dirichlet prior)
% with discrete part, based on a set of observations
%
% Args:
%   ab (int 1-D array): Dirichlet parameters a=ab(1), b=ab(2), or a=b=ab(1) if scalar; a and b must be >0
%
%   obs (int 2-D array): a vector of observations where obs(:,1) is successes and obs(:,2) is total tries
%
%   opts (struct): options, with fields
%
%     - hvec (float 1-D array): vector of weights for point-mass probabilities,  must be all >=0 and sum to <=1 (not checked for), defaults to [];
%     - qvec (float 1-D array): vector of probabilities corresponding to the elements of hvec, must all be strictly >0 and < 1
%
% Returns:
%   loglik (float): log likelihood (natural log)
%   opts_used (struct): option values used
%
% See also: LOGLIK_BETA, BETALN, FILLDEFAULT.
%
if (nargin<3)
    opts=struct();
end
opts=filldefault(opts,'hvec',[]);
opts=filldefault(opts,'qvec',[]);
%
if length(ab)==1
    ab=[ab ab];
end
a=ab(1);
b=ab(2);
nobs=size(obs,1);
%obs is [successes tries]
loglik_beta_each=zeros(nobs,1);
if nobs>0
    a_args=a+obs(:,1);
    b_args=b+obs(:,2)-obs(:,1);
    if any(a_args<=0) | any(b_args<=0)
        loglik_beta_each=Inf;
    else
        loglik_beta_each=betaln(a_args,b_args); %contribution of each non-exceptional beta term
    end
end
loglik_beta=sum(loglik_beta_each);
%
if (a<=0) | (b<=0)
    lognorm_beta_each=-Inf;
else
    lognorm_beta_each=-betaln(a,b);
end
lognorm_beta=lognorm_beta_each*nobs;
%
if sum(opts.hvec)>0
    lik_beta_each=exp(loglik_beta_each+lognorm_beta_each);% beta-component, with normalization
    lik_pointmass_each=ones(nobs,length(opts.hvec));
    for hptr=1:length(opts.hvec)
        q=opts.qvec(hptr);
        lik_pointmass_each(:,hptr)=q.^(obs(:,1)).*(1-q).^(obs(:,2)-obs(:,1)); %contribution of point-mass
    end
    lik_total_each=lik_beta_each*(1-sum(opts.hvec))+lik_pointmass_each*opts.hvec(:); %weighted sum
    loglik=sum(log(lik_total_each));
    lognorm=0;
else
    lognorm=lognorm_beta;
    loglik=loglik_beta;
end
loglik=loglik+lognorm;
%
if any(opts.hvec<0) | (sum(opts.hvec)>1) %put in a wall so that h stays withinb legal range
    loglik=-Inf;
end
opts_used=opts;
return

`mindex_inv(dimranges, indexes)`

[vecs,mults,addoff]=mindex_inv(dimranges,indexes) implements a multi-indexing scheme with ranges are in dimranges

This allows for arbitrary start and stop indices along each dimension. In contrast to mindex_make, the first entry in dimranges is the outer loop, i.e., cycles the slowest.

Input arguments:

dimranges (int 2-D array) –

an array of size [ndims 2], dimranges(k,1) is the lowest index (typically 0 or 1), dimranges(k,2) is the highest index for dimension k
indexes (int 1-D array) –

a column vector of scalars to convert to a multi-index; if not supplied, then the max of each row of dimranges is assumed

Output arguments:

vecs ( int 2-D array ) –

array of size [length(indexes),ndims]: vecs(k,b) is the multi-index converted from indexes(b); each vecs(k,b) is in [dimranges(k,1) dimranges(k,2)]
mults ( int 1-D array ) –

a column of length ndims
addoff ( int ) –

offset to convert from vecs to indices, if supplied; indices=vecs*mults+addoff

`mindex_make(dims)`

[mindex,mults]=mindex_make(dims) creates a multi-indexing scheme for an array whose dimensions are in dims

Input arguments:

dims (int 1-D array) –

length of each dimension

Output arguments:

mindex ( int 2-D array ) –

array of size [prod(dims) length(dims)], in which mindex(:,id) has values from 0 to dims(id)-1, mindex(:,1) cycles most rapidly, and all rows are unique
mults ( int 1-D array ) –

a column of length length(dims), containing [1 dims(1) dims(1)*dims(2) ... dims(1)*...*dims(end-1)]. mindex*mults is [0:prod(dims)-1]

See also: mindex_inv, mindex_make_check.

Source code in src/utils/mindex_make.m

function [mindex,mults]=mindex_make(dims)
% [mindex,mults]=mindex_make(dims) creates a multi-indexing scheme for an array whose dimensions are in dims
%
% Args:
%   dims (int 1-D array): length of each dimension
%
% Returns:
%   mindex (int 2-D array): array of size [prod(dims) length(dims)], in which mindex(:,id) has values from 0 to dims(id)-1, mindex(:,1) cycles most rapidly, and  all rows are unique
%
%   mults (int 1-D array): a column of length length(dims), containing [1 dims(1) dims(1)\*dims(2) ... dims(1)\*...\*dims(end-1)]. mindex\*mults is [0:prod(dims)-1]
%    
% See also: MINDEX_INV, MINDEX_MAKE_CHECK.
%
r=length(dims);
dprod=[1 cumprod(dims)];
mults=dprod(1:end-1)';
mindex=zeros(prod(dims),r);
for id=1:r
    mindex(:,id)=floor(mod([0:dprod(end)-1]',dprod(id+1))/dprod(id));
end
return

`mindex_make_check(tags, opts)`

[opts_used,mindex,mults]=mindex_make_check(tags,opts) creates or checks a multi-index computed by mindex_make

Input arguments:

tags (int or float array) –

target for multi-indexing: a multidimensional array (typically of integers), must have dimension >=2
opts (struct) –
options, with fields
- mindex (int 2-D array) – multi-index array, from mindex_make, computed if not supplied
- mults (int 1-D array) – multiplier from mindex_make (computed if not supplied)

Output arguments:

opts_used ( struct ) –

opts, with mindex, mults computed
mindex ( int 2-D array ) –

opts_used.mindex
mults ( int 1-D array ) –

opts_used.mults

`multi_shuff_enum(counts, opts)`

[a,opts_used]=multi_shuff_enum(counts,opts) enumnerates all the ways of ordering counts(1) 1's, counts(2) 2's, etc.

The algorithm is recursive.

Input arguments:

counts (int 1-D array) –

how many 1's, 2's, 3's, etc. Only non-negative integer entries
opts (struct) –
structure, may be omitted, with fields
- if_log (int) – 1 to log; default is 0
- level (int) – tracks recursion level; default is 0
- if_reduce: 1 to reduce by symmetry if any of counts are identical; 0 to omit; default is 0

Output arguments:

a ( int 2-D array ) –

a list of the shuffles, each row contains counts(1) 1's, counts(2) 2's, etc
opts_used ( struct ) –

options used

Multiplicities

The number of rows in a, dim(a,1) is sum(counts)!/prod(counts(k)!)/r, a multinomial coefficient possibly divided by r, if if_reduce=1, where r is a product of the factorials of the multiplicities of the counts. For example, with counts=[2 3 3 2 2 2], there are 4 tokens that appear twice (1,4,5, and 6), and 2 tokens that appear three times (2 and 3), so r=4!*2!

counts=[2 3 3 2 2 2];

tic;[q,ou]=multi_shuff_enum(counts,setfields(struct(),{'if_log','if_reduce'},{0 1}));toc

Elapsed time is 4.552206 seconds.

disp(size(q))

3153150 14

disp(factorial(sum(counts))/prod(factorial(counts))/24/2)

3153150

disp(q([1:500000:size(q,1)],:))

1 1 2 2 2 3 3 3 4 4 5 5 6 6

2 1 2 4 3 5 1 3 2 5 6 3 4 6

2 2 3 1 4 5 3 4 3 6 1 5 6 2

1 4 2 3 1 4 2 5 6 2 3 6 3 5

2 1 4 2 4 5 2 6 5 3 3 6 1 3

1 4 5 2 3 3 6 4 2 2 6 1 3 5

1 4 5 2 6 2 1 5 4 3 3 6 2 3

`multi_shuff_groups(gps, opts)`

[shuffs,gp_info,opts_used]=multi_shuff_groups(gps,opts) creates lists of shuffles between groups, that optionally:

Optionally, the shuffles can be forced to preserve an auxiliary tag (so that only elements with the same tag are exchanged).

Optionally, the list of shuffles can be exhaustive.

Optionally, if exhaustive, the list of shuffles can be reduced by symmetry so that shuffles that differ only by the order of the groups are not considered different.

The algorithmn for auxiliary tags relies on recursion, but recursion only one step deep.

Input arguments:

gps (int 1-D array) –

a row vector of length n, that indicates the assignment of each element to [1:ngps]. Not all of [1:ngps] must be present.
opts (struct) –
structure, may be omitted, with fields
- if_log (int) – 1 to log; default is 0
- if_ask (int) – 1 to ask if_reduce, if_exhaust, nshuffs, -1: ask only about if_exhaust and nshuffs; default is 0
- if_exhaust (int) – 1 to attempt to do exhaustive list, 0 for random shuffles; defaults is 0
- if_reduce (int) – 1 to reduce by symmetrizing, defaults is 0; ignored if if_exhaust=0
- if_justcount (int) – 1 to count number of shuffles but not compute them; 0 also computes; defaults is 0
- if_nowarn (int) – 1 to suppress warnings; defaults is 0
- if_sortrows (int) – 1 to sort shuffle list by rows; default is 0
- nshuffs (int) – number of shuffles if random shuffles are used, default is 1000
- exhaust_raw_max (int) – maximum number of shuffles if exhaustive list is requested, before reduction by symmetry; defaults is 10^9
- exhaust_reduced_max (int) – maximum number of shuffles if exhaustive list is requeted, after reduction by symmetry; default is 10^6
- nshuffs_max (int) – maximum number of shuffles if exhaustive list is requete3d, default is min(exhaust_raw_max,exhaust_reduced_max)
- tags (int 1-D array) – row vector of length n, containing values [1:ntags]; shuffles will only exchange values with the same tags. Ignored if empty or all values equal. Not all of [1:ntags] need be represented. Defaults to ones(1,n)

Output arguments:

shuffs ( int 2-D array ) –

the shuffles; each row is a permutation of [1:n]
gp_info ( struct ) –
group information, with fields
- ngps (int) – number of groups
- gps_unique (int 1-D array) – unique group numbers, in order
- gp_list (cell array) – gp_list{igp} which elements in each group, with igp a pointer into gps_unique
- nsets_gp (int 1-D array) – number of datasets in each group, in order of gps_unique
- exhaust_raw (int) – number of shuffles if exhaustive, prior to reduction by symmetry
- exhaust_reduced (int) – number of shuffles if exhaustive, after reduction by symmetry
- tags (cell array of struct) – structure with field 'tag' indicating the tag numnber, and field 'gp_info' with gp_info for the tag
opts_used ( struct ) –

opts, with defaults filled in

Reduction by symmetry

If if_reduce=1, then groups of the same size, and that contain the same number of items for each tag, are considered identical and not separatelyh listed.
That is, for every set of groups that have the same number of items with each tag, the groups are ordered in order of the lowest-occurring element within each group.
Groups that have the same size but differ in the number of elements that they contain of each tag are considered different. For example, if gps=[1 1 1 2 2 2] and tags=[1 1 2 2 2 1], then items [1 2 6] can be permuted (tag=1), and independently items [3 4 5] can be permuted (tag=2), but even though both groups have 3 elements, the first group is distinguished since it has two elements with a tag of 1. But if gps=[1 1 1 2 2 2] and tags=[1 1 2 1 1 2], then there are 6 rearrangements of items [1 2 4 5], and 2 rearrangements of items [3 6], for a total of 12=6*2 shuffles into two groups of 3. However, with if_reduce=1, these are reduced by a factor of two since the first and second groups both have the same complements of each tag.

`nary2int(nary, d, lastdim)`

ints=nary2int(nary,d,lastdim) converts one or more vectors, considered as integers base d, to integers

One value of ints is calculated for each element in the first ndims(nary)-1 dimensions of nary. The last dimension of nary is the base-d expansion, with the least significant bit in position 1.

Input arguments:

nary (int array) –

an array of n-ary numbers
d (int or int array) –

the base, 2 if omitted, d may be a vector; see note below regarding generalized n-ary conversion
lastdim (int) –

last dimension of nary; default is ndims(nary). Required if, for example, size(nary)=[10 7] but this is thought of as [10 7 1]

Output arguments:

ints ( int or int array ) –

the converted integers.

Generalized n-ary conversion

If d is a vector, then d(1) is used for the first place, d(2) for the second, etc. and d(end) used if d does not have sufficient length.

In this case, ints(k)=nary(k,1)+d(1)*nary(k,2)+d(1)*d(2)*nary(k,3)+...

`nicesubp(nsubplots, aspect)`

[nr,nc,aused]=nicesubp(nsubplots,aspect) finds a nice number of rows and columns to put nsubplots into a figure

Input arguments:

nsubplots (int) –

number of subplots to position
aspect (float) –
desired aspect ratio of overall plot; default is 1
- If a scalar, the height/width ratio
- if [v1,v2], the allowed range

Output arguments:

nr ( int ) –

number of rows in subplot array
nc ( int ) –

number of columns in subplot array
aused ( float ) –

aspect ratio used

Source code in src/utils/nicesubp.m

function [nr,nc,aused]=nicesubp(nsubplots,aspect)
% [nr,nc,aused]=nicesubp(nsubplots,aspect) finds a nice number of rows and columns to put nsubplots into a figure
%
% Args:
%   nsubplots (int): number of subplots to position
%
%   aspect (float): desired aspect ratio of overall plot; default is 1
%   
%     - If a scalar, the height/width ratio
%     - if [v1,v2], the allowed range
%
% Returns:
%   nr (int): number of rows in subplot array
%
%   nc (int): number of columns in subplot array
%
%   aused (float): aspect ratio used
%
if (nargin<=1) aspect=1; end
%
if (nsubplots<=1)
   nr=1;nc=1;
   return
end
if (length(aspect)==1)
   avals=aspect;
else
   avals=[aspect(1):(aspect(2)-aspect(1))/(sqrt(nsubplots)*2):aspect(2)]; % a range of values
   %reorder in terms of decreasing distance to midpoint
   dvals=abs(avals-(aspect(1)+aspect(2))/2);
   [sdvals,isort]=sort(-dvals);
   avals=avals(isort);
end
%
missing=inf;
for aval=avals
   nc=sqrt(nsubplots/aval);
	nr=nc*aval;
	if (aval<=1)
   	nc=ceil(nc*(1-0.5/max(nc,nr)));
   	nr=ceil(nsubplots/nc);
	else
  		nr=ceil(nr*(1-0.5/max(nc,nr)));
   	nc=ceil(nsubplots/nr);
   end
   if ((nr*nc-nsubplots)<=missing); %best yet?
      nrb=nr;
      ncb=nc;
      missing=nr*nc-nsubplots;
      aused=nr/nc;
   end
end
nr=nrb;
nc=ncb;

`overlap_heuristics(overlaps)`

h=overlap_heuristics(overlaps) computes several heuristics for whether the number of overlaps of pairwise distances across several datasets suffices for a well-defined knitting across these datasets

Input arguments:

overlaps (int 2-D array) –

overlaps(istim,iset) is 1 if stimulus istim is present in record iset, 0 otherwise

Output arguments:

h ( struct ) –
structure of heuristics, with fields
- nstims (int) – number of stimuli (size(overlaps,1))
- nstims_used (int) – number of stimuli present in at least one record
- npairs (int) – total number of pairwise distances within records
- nmax (int) – maximum number of pairwise distances if all stimuli were in the same record
- counts (int 2-D array) – counts(istim,jstim) is the number of records that contain both stimuli
- nfree (int 1-D array) – nfree(iset) is the number of stimuli unique to record iset
- nbound (int 1-D array) – nbound(iset) is the number of stimuli in record iset that are in at least one other record
- novlp (int 2-D array) – novlp(iset,jset) is the number of stimuili in common between records iset and jset
- dmax_free (int) – maximum dimension that is rigidly constrained by bound points, equal to min(nbound-1) for all records with nfree(irec)>0
- dmax_constraints (int) – maximum dimension for which number of coordinates to be found, minus offset and rotational d.o.f., does not exceed npairs
- dmax (int) – maximum dimension for which a well-defined knitting solution can be expected, min(dmax_free,dmax_constraints)

Note regarding format of overlaps

The format of overlaps is identical to the optional argument opts_pcon.overlaps in procrustes_consensus.

`pairs2chains(pairs)`

chains=pairs2chains(pairs) converts pairs of tokens to chains of tokens

For example, the pairs [a b;c d;a e;a f;c f] are converted to the chains {[a b],[e a f],[d c f]}. The pairs are un-ordered, and must be unique. This is useful for graphics: rather than connect multiple pairs of points with each pair connected as separate line objects, one can instead connect them with a smaller number of line objects, one for each chain.

This likely could be optimized; here the algorithm is just to extend chains until there is no unused node. There is no attmept to merge chains, e.g., by splicing together intersecting loops.

Input arguments:

pairs (int 2-D array) –

a 2-column array of the pairs; each token is a unique integer

Output arguments:

chains ( cell array ) –

a cell array of vectors of chains; paired tokens are adjacent in exactly one chain

Source code in src/utils/pairs2chains.m

function chains=pairs2chains(pairs)
% chains=pairs2chains(pairs) converts pairs of tokens to chains of tokens
%
% For example, the pairs [a b;c d;a e;a f;c f] are converted to the chains  {[a b],[e a f],[d c f]}.
% The pairs are un-ordered, and must be unique. This is useful for graphics: rather than connect multiple pairs of points with each pair connected as separate line objects,
% one can instead connect them with a smaller number of line objects, one for each chain.
%
% This likely could be optimized; here the algorithm is just to extend chains until there is no unused node.
% There is no attmept to merge chains, e.g., by splicing together intersecting loops.
% 
% Args:
%   pairs (int 2-D array): a 2-column array of the pairs; each token is a unique integer
%
% Returns:
%   chains (cell array): a cell array of vectors of chains; paired tokens are adjacent in exactly one chain
% 
chains=cell(0);
loop_list=[];
%
npairs=size(pairs,1);
tmax=max(pairs(:));
tmin=min(pairs(:));
pairs=pairs-tmin+1;
mtx=zeros(tmax-tmin+1);
%create an incidence matrix
for ip=1:npairs
    mtx(pairs(ip,1),pairs(ip,2))=1;
end
mtx=mtx+mtx';
npairs_used=0;
%choose any unused node and attempt to grow it until the chain returns toitself, or has no choice
while any(sum(mtx,1)>0) %any pairs remaining?
    k=min(find(sum(mtx,1)>0));
    knbrs=find(mtx(k,:)>0); %determine the neigbhbors of k
    nnbrs=length(knbrs);
    if nnbrs==1
        kset=[k knbrs(1)];
    else
        kset=[knbrs(2) k knbrs(1)];
    end
    if_extend=1;
    while if_extend==1 %attempt to extend forward
        ke=kset(end);
        %find k_used: the points of kset that have already been connected with ke
        e_pos=find(kset==ke);
        v_fwd=kset(e_pos(e_pos<length(kset))+1);
        v_rev=kset(e_pos(e_pos>1)-1);
        v_used=union(v_fwd,v_rev);
        kex=setdiff(find(mtx(kset(end),:)>0),v_used); %extend into a new element?
        if isempty(kex)
            if_extend=0;
        else
            kset=[kset,kex(1)];
        end
    end
    if_loop=0;
    if nnbrs>=2
        if mtx(kset(end),kset(1))>0 %found a loop
            if_loop=1;
        else
            if_extend=1;
            while if_extend==1 %attempt to extend backward
                ke=kset(1);
                e_pos=find(kset==ke);
                v_fwd=kset(e_pos(e_pos<length(kset))+1);
                v_rev=kset(e_pos(e_pos>1)-1);
                v_used=union(v_fwd,v_rev);
                kex=setdiff(find(mtx(kset(1),:)>0),v_used); %extend backward?
                if isempty(kex)
                    if_extend=0;
                else
                    kset=[kex(1),kset];
                end
            end
            if mtx(kset(end),kset(1))>0
                if_loop=1;
            else
                if_loop=-1;
            end
        end
    else
        if_loop=-1; %done but no loop
    end
    nk=length(kset);
    chain=kset;
    if if_loop==1
        chain(end+1)=chain(1);
    end
    chains{end+1}=chain+tmin-1; %add offset
    lc=length(chain);
    for icl=1:lc-1
        mtx(chain(icl),chain(icl+1))=0;
        mtx(chain(icl+1),chain(icl))=0;
    end
end
return
end

`persp_apply(a, b, c, x)`

y=persp_apply(a,b,c,x) applies a perspective (projective) transformation to one or more vectors

Input arguments:

a (float 2-D array) –

array of size [dimx, dimy], affine component of the transformation
b (float 1-D array) –

array of size [ 1, dimy], offset component of the transformation
c (float 1-D array) –

array of size [dimx, 1], perspective component of the transformation
x (float 2-D array) –

array of size [npts, dimx], the (row) vectors to be transformed; npts is the number of vectors to be transformed

Output arguments:

y ( float 2-D array ) –

array of size [npts, dimy], its row vectors are the results of transformation

How the transformation is defined

Each row of x is considered as a homogeneous vector with an augmented coordinate 1 at the end, creating an array of size [npts, dimx+1].

A matrix T, of size [dimx+1, dimy+1] is formed with a in its upper left, b in its lower left, c in its upper right, and 1 in its lower right.

X is then post-multiplied by T, Y=XT.

Y is a homogeneous vector. y is the rows of Y, with each row divided by the final element.

T only matters up to homogeneity, but its lower right element is fixed at 1.

If c is zero, this is an affine transformation.

If c and b are zero, this is a linear transformation.

Source code in src/utils/persp_apply.m

function y=persp_apply(a,b,c,x)
% y=persp_apply(a,b,c,x) applies a perspective (projective) transformation to one or more vectors
%
% Args:
%   a (float 2-D array): array of size [dimx, dimy], affine component of the transformation
%
%   b (float 1-D array): array of size [   1, dimy], offset component of the transformation
%
%   c (float 1-D array): array of size [dimx,    1], perspective component of the transformation
%
%   x (float 2-D array): array of size [npts, dimx], the (row) vectors to be transformed; npts is the number of vectors to be transformed
%
% Returns:
%   y (float 2-D array): array of size [npts, dimy], its row vectors are the results of transformation
%
% Note: How the transformation is defined
%   Each row of x is considered as a homogeneous vector with an augmented coordinate 1 at the end, creating an array of size [npts, dimx+1].
%
%   A matrix T, of size [dimx+1, dimy+1] is formed with a in its upper left, b in its lower left, c in its upper right, and 1 in its lower right.
%
%   X is then post-multiplied by T, Y=XT.
%
%   Y is a homogeneous vector.  y is the rows of Y, with each row divided by the final element.
%
%   T only matters up to homogeneity, but its lower right element is fixed at 1.
%
%   If c is zero, this is an affine transformation.
%
%   If c and b are zero, this is a linear transformation.
%
% See also:  PERSP_XFORM_FIND, PERSP_SSQDIF, PERSP_FIT, PERSP_SSQDIF_FIT.
%
T=[a c;b 1];
X=[x,ones(size(x,1),1)];
Y=X*T;
y=Y(:,1:end-1)./repmat(Y(:,end),1,size(a,2));
return
end

`persp_fit(c, x, y)`

[a,b]=persp_fit(c,x,y) fits a projective transformation, assuming a vector for the perspective component

Transformation parameters a,b, and c are described in persp_apply.

Input arguments:

c (float 1-D array) –

array of size [dimx, 1], perspective component of the transformation
x (float 2-D array) –

array of size [npts, dimx], the (row) vectors to be transformed; npts is the number of vectors to be transformed
y (float 2-D array) –

array of size [npts, dimy], its row vectors are to be fit by applying the transformation to x

Output arguments:

a ( float 2-D array ) –

array of size [dimx, dimy], affine component of the transformation
b ( float 1-D array ) –

array of size [ 1, dimy], offset component of the transformation

Source code in src/utils/persp_fit.m

function [a,b]=persp_fit(c,x,y)
% [a,b]=persp_fit(c,x,y) fits a projective transformation, assuming a vector for the perspective component
% 
% Transformation parameters a,b, and c are described in `persp_apply`.
%
% Args:
%   c (float 1-D array): array of size [dimx,    1], perspective component of the transformation
%
%   x (float 2-D array): array of size [npts, dimx], the (row) vectors to be transformed; npts is the number of vectors to be transformed
%
%   y (float 2-D array): array of size [npts, dimy], its row vectors are to be fit by applying the transformation to x
%
% Returns:
%   a (float 2-D array): array of size [dimx, dimy], affine component of the transformation
%
%   b (float 1-D array): array of size [   1, dimy], offset component of the transformation
%
% See also:  PERSP_XFORM_FIND, PERSP_APPLY, PERSP_SSQDIF, PERSP_SSQDIF_FIT.
%
denom=x*c+1;
regressors=[x./repmat(denom,1,size(x,2)) 1./denom];
a=zeros(size(x,2),size(y,2));
b=zeros(1,size(y,2));
for iy=1:size(y,2)
    ab=regress(y(:,iy),regressors);
    a(:,iy)=ab(1:end-1);
    b(1,iy)=ab(end);
end
return

`persp_ssqdif(a, b, c, x, y)`

[ssq,y_fit]=persp_ssqdif(a,b,c,x,y) determines the deviation between a projective transformation and target values

Transformation parameters a,b, and c are described in persp_apply.

Input arguments:

a (float 2-D array) –

array of size [dimx, dimy], affine component of the transformation
b (float 1-D array) –

array of size [ 1, dimy], offset component of the transformation
c (float 1-D array) –

array of size [dimx, 1], perspective component of the transformation
x (float 2-D array) –

array of size [npts, dimx], the (row) vectors to be transformed; npts is the number of vectors to be transformed
y (float 2-D array) –

array of size [npts, dimy], its row vectors are to be fit by applying the transformation to x

Output arguments:

ssq ( float ) –

sum of squared differences between y and y_fit
y_fit ( float 2-D array ) –

array of size [npts, dimy], the fitted values, i.e., the results of applying the fitted transformation to x

Source code in src/utils/persp_ssqdif.m

function [ssq,y_fit]=persp_ssqdif(a,b,c,x,y)
% [ssq,y_fit]=persp_ssqdif(a,b,c,x,y) determines the deviation between a projective transformation
% and target values
%
% Transformation parameters a,b, and c are described in `persp_apply`.
%
% Args:
%   a (float 2-D array): array of size [dimx, dimy], affine component of the transformation
%
%   b (float 1-D array): array of size [   1, dimy], offset component of the transformation
%
%   c (float 1-D array): array of size [dimx,    1], perspective component of the transformation
%
%   x (float 2-D array): array of size [npts, dimx], the (row) vectors to be transformed; npts is the number of vectors to be transformed
%
%   y (float 2-D array): array of size [npts, dimy], its row vectors are to be fit by applying the transformation to x
%
% Returns:
%   ssq (float): sum of squared differences between y and y_fit
%
%   y_fit (float 2-D array): array of size [npts, dimy], the fitted values, i.e., the results of applying the fitted transformation to x
%
% See also:  PERSP_XFORM_FIND, PERSP_APPLY, PERSP_FIT, PERSP_SSQDIF_FIT.
%
y_fit=persp_apply(a,b,c,x);
ssq=sum((y(:)-y_fit(:)).^2);
return
end

`persp_ssqdif_fit(c, x, y)`

[ssq,y_fit,a,b]=persp_ssqdif_fit(c,x,y) fits a projective transformation, assuming a perspective component, and finds the deviation between target and fitted values

Transformation parameters a,b, and c are described in persp_apply.

Input arguments:

c (float 1-D array) –

array of size [dimx, 1], perspective component of the transformation
x (float 2-D array) –

array of size [npts, dimx], the (row) vectors to be transformed; npts is the number of vectors to be transformed
y (float 2-D array) –

array of size [npts, dimy], its row vectors are to be fit by applying the transformation to x

Output arguments:

ssq ( float ) –

sum of squared differences between y and y_fit
y_fit ( float 2-D array ) –

array of size [npts, dimy], the fitted values, i.e., the results of applying the fitted transformation to x
a ( float 2-D array ) –

array of size [dimx, dimy], affine component of the transformation
b ( float 1-D array ) –

array of size [ 1, dimy], offset component of the transformation

Source code in src/utils/persp_ssqdif_fit.m

function [ssq,y_fit,a,b]=persp_ssqdif_fit(c,x,y)
% [ssq,y_fit,a,b]=persp_ssqdif_fit(c,x,y) fits a projective transformation, assuming a perspective component, 
% and finds the deviation between target and fitted values
%
% Transformation parameters a,b, and c are described in `persp_apply`.
%
% Args:
%   c (float 1-D array): array of size [dimx,    1], perspective component of the transformation
%
%   x (float 2-D array): array of size [npts, dimx], the (row) vectors to be transformed; npts is the number of vectors to be transformed
%
%   y (float 2-D array): array of size [npts, dimy], its row vectors are to be fit by applying the transformation to x
%
% Returns:
%   ssq (float): sum of squared differences between y and y_fit
%
%   y_fit (float 2-D array): array of size [npts, dimy], the fitted values, i.e., the results of applying the fitted transformation to x
%
%   a (float 2-D array): array of size [dimx, dimy], affine component of the transformation
%
%   b (float 1-D array): array of size [   1, dimy], offset component of the transformation
%
% See also:  PERSP_XFORM_FIND, PERSP_APPLY, PERSP_FIT, PERSP_SSQDIF.
%
[a,b]=persp_fit(c,x,y);
[ssq,y_fit]=persp_ssqdif(a,b,c,x,y);
return
end

`persp_xform_find(x, y, opts)`

[persp,y_fit,opts_used]=persp_xform_find(x,y,opts) fits a projective transformation

Input arguments:

x (float 2-D array) –

array of size [npts, nd], the (row) vectors to be transformed; npts is the number of vectors to be transformed
y (float 2-D array) –

array of size [npts, nd], its row vectors are the to be fit by applying the transformatoin to x
opts (struct) –
an options structure, may be omitted, with fields
- method (char) – optimization method to use, can be 'fmin','oneshot','best'; default is 'oneshot' but this is overridden to 'best' when used by rs software; see note below regarding optimization methods
- if_cycle (int) – relevant if opts.method='oneshot' or 'best'; see note below regarding optimization methods
- fmin_opts (struct) – relevant if opts_method='fmin' or 'best; see note below regarding optimization methods

Output arguments:

persp ( float 2-D array ) –

array of size [nd+1 nd+1] specifying the transformation; persp*[x ones(npts,1)] are the homogeneous coordinates cooresponding to y
y_fit ( float 2-D array ) –

array of size [npts nd], the fitted values, i.e., the mapping of x via persp (see persp_apply for further details)
opts_used ( struct ) –
options used, along with
- ssq (float) – sum of squares of deviations of y_fit from y
- oneshot (struct) – detailed results from 'oneshot' method
- fmin (struct) – detailed results from 'fmin' method

Optimization methods

'fmin': an iterative method: An initial specifier of the perspective component (c in persp_apply) is guessed. Based on c, the other components of the transformation (a and b in persp_apply) are determined by linear regresssion, yielding a squared-error residual. The residual is then minimized by adjusting c via MATLAB's fmin, which is called with the options in opts.fmin_opts
'oneshot': This is an implementation of method 2 of The method Z. Zhang, Estimating Projective Transformation Matrix (Collineation, Homography), Microsoft Research Techical Report MSR-TR-2010-63, November 1993; Updated May 29, 2010. It is most apprpriate when the transformation is highly overdetermined, i.e, when there are a large number of data points. If opts.if_cycle=1 (default is 0), this cycles through assignments of which data point is treated as the first point; the one that yields the best fit is returned in opts_used.best_point.
'best': Methods 'fmin' and 'oneshot' are both are used, and the results of the method that yields the smallest squared-error residuals is returned

Source code in src/utils/persp_xform_find.m

function [persp,y_fit,opts_used]=persp_xform_find(x,y,opts)
% [persp,y_fit,opts_used]=persp_xform_find(x,y,opts) fits a projective transformation
%
% Args: 
%   x (float 2-D array): array of size [npts, nd], the (row) vectors to be transformed; npts is the number of vectors to be transformed
%
%   y (float 2-D array): array of size [npts, nd], its row vectors are the to be fit by applying the transformatoin to x
%
%   opts (struct): an options structure, may be omitted, with fields
%
%     - method (char): optimization method to use, can be 'fmin','oneshot','best'; default is 'oneshot' but this is overridden to 'best' when used by rs software; see note below regarding optimization methods
%     - if_cycle (int): relevant if  opts.method='oneshot' or 'best'; see note below regarding optimization methods
%     - fmin_opts (struct): relevant if opts_method='fmin' or 'best; see note below regarding optimization methods
%
% Returns:
%   persp (float 2-D array): array of size [nd+1 nd+1] specifying the transformation; persp*[x ones(npts,1)] are the homogeneous coordinates cooresponding to y
%
%   y_fit (float 2-D array): array of size [npts nd], the fitted values, i.e., the mapping of x via persp (see `persp_apply` for further details)
%
%   opts_used (struct): options used, along with
%
%     - ssq (float): sum of squares of deviations of y_fit from y
%     - oneshot (struct): detailed results from 'oneshot' method
%     - fmin (struct): detailed results from 'fmin' method
%
% Note: Optimization methods
%   - 'fmin': an iterative method: An initial specifier of the perspective component (c in `persp_apply`) is guessed. Based on c, 
%   the other components of the transformation (a and b in `persp_apply`) are determined by linear regresssion,
%   yielding a squared-error residual. The residual is then minimized by adjusting c via MATLAB's fmin, which is called with the options in opts.fmin_opts
%   - 'oneshot': This is an implementation of method 2 of The method Z. Zhang, Estimating Projective Transformation Matrix (Collineation, Homography),
%   Microsoft Research Techical Report MSR-TR-2010-63, November 1993;
%   Updated May 29, 2010.  It is most apprpriate when the transformation
%   is highly overdetermined, i.e, when there are a large number of data
%   points. If opts.if_cycle=1 (default is 0), this cycles through assignments of which data point is treated as the first point; the one that yields the best fit is returned in opts_used.best_point. 
%   - 'best': Methods 'fmin' and 'oneshot' are both are used, and the results of the method that yields the smallest squared-error residuals is returned
%
% See also: FILLDEFAULT, PERSP_SSQDIF, PERSP_SSQDIF_FIT.
%
if (nargin<=2) opts=struct; end
opts=filldefault(opts,'method','oneshot');
opts=filldefault(opts,'if_cycle',0);
opts=filldefault(opts,'fmin_opts',struct());
opts_used=opts;
%
npts=size(x,1);
nd=size(x,2);
if (npts<=nd+1)
    warning(sprintf('perspective fitting is underdetermined.  need npts>=nd+2, nd=%3.0f npts=%3.0f',nd,npts));
end
switch opts.method   
    case 'oneshot'
        [persp,y_fit,opts_used]=persp_xform_find_oneshot(x,y,opts_used);
    case 'fmin'
        [persp,y_fit,opts_used]=persp_xform_find_fmin(x,y,opts_used);
    case 'best'
        z=struct;
        [z.oneshot.persp,z.oneshot.y_fit,z.oneshot.opts_used]=persp_xform_find_oneshot(x,y,opts_used);
        [z.fmin.persp,z.fmin.y_fit,z.fmin.opts_used]=persp_xform_find_fmin(x,y,opts_used);
        if z.fmin.opts_used.ssq<=z.oneshot.opts_used.ssq
            method_best='fmin';
        else
            method_best='oneshot';
        end
        opts_used.method_best=method_best;
        opts_used.oneshot=z.oneshot;
        opts_used.fmin=z.fmin;
        persp=z.(method_best).persp;
        y_fit=z.(method_best).y_fit;
        opts_used.ssq=z.(method_best).opts_used.ssq;
end %opts.method       
return
end

`procrustes_compat(t)`

t_new=procrustes_compat(t) converts the fieldnames produced by procrustes_consensus to names compatible with transformation structures

Input arguments:

t (struct) –

a structure with fields 'scaling', 'orthog', and 'translation'

Output arguments:

t_new ( struct ) –

a structure with fields renamed 'b','T','c'

Default values

If 'scaling' is unspecified in 't', b is set to 1 in t_new.

If 'translation' is unspecified in t, 'c' is set to zeros(size(orthog,2)) in t_new

Source code in src/utils/procrustes_compat.m

function t_new=procrustes_compat(t)
% t_new=procrustes_compat(t) converts the fieldnames produced by `procrustes_consensus` to names compatible with `transformation structures`
%
% Args:
%   t (struct): a structure with fields 'scaling', 'orthog', and 'translation'
%
% Returns:
%   t_new (struct): a structure with fields renamed 'b','T','c'
%
% Note: Default values
%   If 'scaling' is unspecified in 't', b is set to 1 in t_new.
% 
%   If 'translation' is unspecified in t, 'c' is set to zeros(size(orthog,2)) in t_new
%
% See also:  FILLDEFAULT, PROCRUSTES_CONSENSUS.
%
t=filldefault(t,'scaling',1);
t=filldefault(t,'translation',zeros(1,size(t.orthog,2)));
%
t_new=struct;
t_new.T=t.orthog;
t_new.b=t.scaling;
t_new.c=t.translation;
%
return
end

`procrustes_consensus(z, opts_pcon)`

[consensus,znew,ts,details,opts_pcon_used]=procrustes_consensus(z,opts_pcon) carries out a Procrustes consensus calculation

This algorithm generalizes MATLAB's procrustes tomore than two datasets, allowing for missig values (indicated by NaNs), eliminating translations, and creation of a consensus that is of higher dimension than any of the components, by zero-padding z along dimension 2.

If called with no arguments, the default settings for opts_pcon are returned.

Input arguments:

z (float 3-D array) –

input data, size is [npts nds nsets] where npts is the number of points, nds is the number of dimensions, and nsets is the number of datasets
opts_pcon (struct) –
options struture, may be omitted, with fields

Output arguments:

consensus ( float 2-D array ) –

[npts ndims]: the consensus data
znew ( float 3-D array ) –

[npts ndims nsets]: the original data, after transformation to the consensus
ts ( cell 1-D array ) –

the transformations. They act as follows: znew(:,:,iset)=ts{iset}.scaling*z(:,:,iset)*ts{iset}.orthog+repmat(ts{iset}.translation,npts,1). These variable names can be converted to MATLAB's conventions via procrustes_compat.
details ( struct ) –
details of convergence, with fields
- ts_cum{k}{iset} is the cumulative transformation found for dataset iset at iteration k
- consensus(:,:,k) is the consensus found at iteration k
- z(:,:,iset,k) is the best fit of each dataset at iteration k
- rms_change(k) is the rms change at iteration k
- rms_dev(iset,k) is the rms deviation between all fitted points in dataset iset and the consensus at iteration k (points without overlaps are omitted), per coordinate
- zz_check_diff (k+1,iset) – , are computatooinal checks at each iteration, should be neaer zero
- warnings: warnings if not enough overlaps
- overlap_pairs (iset1,iset2) – number of points that overlap between datasets 1 and 2
- overlap_totals (ipt) – number of overlaps at each data point
- initialize_use (ipt,1) – which dataset is used to initialize each point (if initialize_set>0)
opts_pcon_used ( struct ) –

opts_pcon, with options used filled in

The algorithm

Initialization: guess a consensus via one of the methods specified by initialize_set

Iteration: - Align all the datasets to the tentative consensus, using settings in opts_pcon for scale, reflection, and offset - Set the new tentative consensus to be the point-by-point centroid of each of the component dataasets, following the above alignment - Align the tentative consensus as specified by opts_pcon.alignment, to prevent drift

Terminate after coordinates have not changed within tolerance of opts_pcon.max_rmstol

As a screen for whether there is sufficient overlap between the datasets, isgraphc is used; if the overlap graph is disconnected, a consensus alignment should not be attempted. But note that connectivity of the overlap graph is not sufficient to guarantee stability, as multiple points (depending on the dimension to be fit, and the number of datasets that overlap) are needed to determine a transformation. Overlap details are in details.overlap_pairs and details.overlap_totals .

`rmsubfield(s, fns)`

r=rmsubfield(s,fns) removes a (possibly deeply) nested subfield

Input arguments:

s (struct) –

a structure
fns (cell array) –

field names, fns={n₁, n₂,..., n_k}

Output arguments:

r ( struct ) –

s, with s.(n₁).(n₂). ... .(n_k) removed

Notes

If any of the subfields are not present, r=s.

Subfield arrays not supported.

Uses recursion:

k>=2: rmsubfield(s,{n₁, n₂,..., n_k})=setfield(s.(n₁),rmsubfield(s.(n₁),{n₂,..., n_k}))
k=1: rmsubfield(s,{n₁, n₂,..., n_k})=rmfield(s,n₁)

See also: SETFIELD, RMFIELD, getsubfield.

Source code in src/utils/rmsubfield.m

function r=rmsubfield(s,fns)
% r=rmsubfield(s,fns) removes a (possibly deeply) nested subfield
%
% Args:
%   s (struct): a structure
%
%   fns (cell array): field names, fns={n<sub>1</sub>, n<sub>2</sub>,..., n<sub>k</sub>}
%
% Returns: 
%   r (struct): s, with  s.(n<sub>1</sub>).(n<sub>2</sub>). ... .(n<sub>k</sub>) removed
%
% Note: Notes
%   If any of the subfields are not present, r=s.
% 
%   Subfield arrays not supported.
%
%   Uses recursion:
%
%     - k>=2: rmsubfield(s,{n<sub>1</sub>, n<sub>2</sub>,..., n<sub>k</sub>})=setfield(s.(n<sub>1</sub>),rmsubfield(s.(n<sub>1</sub>),{n<sub>2</sub>,..., n<sub>k</sub>}))
%     - k=1:  rmsubfield(s,{n<sub>1</sub>, n<sub>2</sub>,..., n<sub>k</sub>})=rmfield(s,n<sub>1</sub>)
%
%  See also:  SETFIELD, RMFIELD, GETSUBFIELD.
%
if isfield(s,fns{1})   
    k=length(fns);
    if k>1
        r=setfield(s,fns{1},rmsubfield(s.(fns{1}),fns(2:end)));
    else
        r=rmfield(s,fns{1});
    end
else
    r=s;
end
return

`setfields(oldstruct, cfields, cvals)`

newstruct=setfields(oldstruct,cfields,cvals) sets multiple fields of a structure

Input arguments:

oldstruct (struct) –

structure, may be empty
cfields (cell array of char) –

field names
cvals (cell array) –

field values

Output arguments:

newstruct ( struct ) –

oldstruct, with each field cfields{k} set equal to cvals{k}

Source code in src/utils/setfields.m

function newstruct=setfields(oldstruct,cfields,cvals)
% newstruct=setfields(oldstruct,cfields,cvals) sets multiple fields of a structure
%
% Args:
%   oldstruct (struct): structure, may be empty
%
%   cfields (cell array of char): field names
%
%   cvals (cell array): field values
%
% Returns:
%   newstruct (struct): oldstruct, with each field cfields{k} set equal to cvals{k}
%
newstruct=oldstruct;
if (length(cfields)~=length(cvals))
   error('number of fields and number of values must match.')
end
for k=1:length(cfields)
   newstruct=setfield(newstruct,cfields{k},cvals{k});
end

`surf_augvec(varargin)`

h=surf_augvec plots a surface, extending MATLAB's surf to cases inwhich the surface to be plotted has a dimension of length 1.

Input arguments:

varargin (float array or sequence of float arrays) –
arguments to MATLAB's surf in one of the following formats
- surf_augvec(x,y,z,c), where x and y are the independent variables as 1-D vectors, z is surface height as 2-D array, c is color as 2-D array
- surf_augvec(x,y,z), as above, with c proportional to surface height
- surf_augvec(z,c), as above, with x=[1:size(z,2)] and y=[1:size(z,1)]
- surf_augvec(z), as above, with x=[1:size(z,2)] and y=[1:size(z,1)] and c proportional to surface height
- or any of the above preceded by a handle of the axis to plot into

Output arguments:

h ( handle ) –

handle to the surface

Source code in src/utils/surf_augvec.m

function h=surf_augvec(varargin)
% h=surf_augvec plots a surface, extending MATLAB's `surf` to cases inwhich the surface to be plotted has a dimension of length 1.
% 
% Args:
%   varargin (float array or sequence of float arrays): arguments to MATLAB's `surf` in one of the following formats
%
%     - surf_augvec(x,y,z,c), where x and y are the independent variables as 1-D vectors, z is surface height as 2-D array, c is color as 2-D array
%     - surf_augvec(x,y,z), as above, with c proportional to surface height
%     - surf_augvec(z,c), as above, with x=[1:size(z,2)] and y=[1:size(z,1)]
%     - surf_augvec(z), as above, with x=[1:size(z,2)] and y=[1:size(z,1)] and c proportional to surface height
%     - or any of the above preceded by a handle of the axis to plot into
% 
% Returns:
%   h (handle): handle to the surface
%
if isgraphics(varargin{1},'axes')
    h=varargin{1};
    nh=1;
else
    h=[];
    nh=0;
end
c=[];
switch length(varargin)-nh
    case 1
        z=varargin{1+nh};
        c=z;
        x=[1:size(z,2)];
        y=[1:size(z,1)];
    case 2
        z=varargin{1+nh};
        c=varargin{2+nh};
        x=[1:size(z,2)];
        y=[1:size(z,1)];
    case 3
        x=varargin{1+nh};
        y=varargin{2+nh};
        z=varargin{3+nh};
        c=z;
    case 4
        x=varargin{1+nh};
        y=varargin{2+nh};
        z=varargin{3+nh};
        c=varargin{4+nh};
end
nx=size(z,2);
ny=size(z,1);
xa=x;
ya=y;
za=z;
ca=c;
nxa=nx;
if nx==1
    za=[nan(ny,1),za,nan(ny,1)];
    ca=[nan(ny,1),ca,nan(ny,1)];
    xa=[xa(1)-1 xa(:)' xa(end)+1];
    nxa=nx+2;
end
if ny==1
    za=[nan(1,nxa);za;nan(1,nxa)];
    ca=[nan(1,nxa);ca;nan(1,nxa)];
    ya=[ya(1)-1 ya(:)' ya(end)+1];
end
h=surf(xa,ya,za,ca);
if (nx==1)
    set(gca,'XLim',x+[-0.5 0.5]);
end
if (ny==1)
    set(gca,'YLim',y+[-0.5 0.5]);
end
%set(gca,'XLim',[min(x(:)),max(x(:))]);
%set(gca,'YLim',[min(y(:)),max(y(:))]);
return

`zpad(ival, ndigits)`

ichars=zpad(ival,ndigits) left-pads a non-negative integer with zeros and creates a string

Input arguments:

ival (int) –

non-negative integer to pad, should be <=10^ndigits
ndigits (int) –

total number of digits for the string

Output arguments:

ichars ( char ) –

the left-padded string

Out-of-range values

No checking is done if ival <0 or >=10^ndigits.

ival<0 will return an empty string.

ival>10^ndigits will be return only the right-most ndigits characters.

Source code in src/utils/zpad.m

function ichars=zpad(ival,ndigits)
%ichars=zpad(ival,ndigits) left-pads a non-negative integer with zeros and creates a string
%
% Args:
%   ival (int): non-negative integer to pad, should be <=10<sup>ndigits</sup>
%
%   ndigits (int): total number of digits for the string
%
% Returns:
%   ichars (char): the left-padded string 
%
% Note: Out-of-range values
%   No checking is done if ival <0 or >=10<sup>ndigits</sup>.
%
%   ival<0 will return an empty string.
%
%   ival>10<sup>ndigits</sup> will be return only the right-most ndigits characters.
%
ich=strcat(repmat('0',1,ndigits-1),sprintf('%d',ival));
ichars=ich([(length(ich)-ndigits+1):length(ich)]);
return