`rs_get_coordsets(fullnames, aux)`

[data_out,aux_out]=rs_get_coordsets(fullnames,aux): reads one or more coordinate files into a dataset structure

This is the preferred method for bringing coordinates from one or more files into a dataset structure suitable for display and geometrical analysis.

For files generated by analysis of similarity judgments via [??how to refer to Python package], this will also import the goodness-of-fit data.
A ray structure will be created.
If only a single data file is to be read, rs_read_coorddata can also be used.
This can also create a coordinate structure based on a quadratic form model, and can augment binary texture domain data by symmetry.
To import coordinates generated externally or that exist as arrays rather than in a file, use rs_import_coordsets.
For non-interactive reading, provide aux.nsets, aux.opts_read.data_fullnames, aux.opts_read.input_type, and set aux.opts_read.if_auto=1.

Input arguments:

fullnames (char or cell array of char) –

full file name with path; if empty, it will be requested interactively. File names must contain the string '_coords'. See note below regarding setup files.
aux (struct) –
a structure, can be omitted, with fields
- nsets (int) – number of records to read, if zero (default), then requested at console, with console option to switch to gui
- opts_read (struct) – can be omitted, with fields listed below
- if_gui (int) – 1 to use graphical interface to get files if file names are not supplied (default), 0 to use console; ignored if nsets=0; see note below regarding customization
- if_log (int) – 1 to log progress, 0 to omit; default is 1; see note below regarding customization
- if_auto (int) – 1 not to ask for confirmations, 0 to ask; default is 0; see note below regarding customization
- nfiles_max (int) – maximum number of files to read; default is 100
- input_type (int or int 1-D array) – 1 for reading a coordinate file, 2 for reading a quadratic form model, 0 to ask; cycled through for each dataset; default is 0; see note below regarding customization
- data_fullname_def (char) – prompt for data file name; see note below regarding customization
- ui_filter (char) – file name filter that appears in graphical user interface when if_gui=1; defaults to '*_coords';see note below regarding customization
- if_warn (int) – 1 to show warnings, 0 to omit; default is 1; see note below regarding customization

Output arguments:

data_out ( struct ) –
dataset structure, containing one records, with fields
- ds (cell array) – coordinate structure, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
- sas (cell array) – stimulus metadata structure, sas{k} is the stimulus metadata for the kth record
- sets (cell array) – set metadata structure, sets{k} is the response metadata for the kth record
aux_out ( struct ) –
auxiliary outputs and parameter values used, with fields
- warnings (char) – warnings generated during consistency check
- warn_bad (int) – number of warnings that prevent further processing
- rayss (cell array of struct) – rayss{k} is the ray structure for the kth record
- opts_read (cell array of struct) – opts_read{k} is aux.opts_read for reading the kth record, with defaults filled in
- opts_rays (cell array of struct) – opts_rays{k} are the options used for creating the ray structure for the kth record
- opts_qpred (cell array of struct) – opts_qpred{k} are the options used for creating the quadratic form model for the kth record along with
  - var_tot (float) – total variance of the coordinates
  - var_ex (float 1-D array) – variance explained by each dimension of the model
  - predcoords (float 2-D array) – predcoords(istim,idim) is the model coordinate for stimulus istim in dimension idim
- opts_check (struct) – aux.opts_check, with defaults filled in
- syms_list (struct) – structure with results of symmetry augmentation
  - sym_apply (char) – the symmetry applied, see 'sym_apply' above
  - primary (int 1-D array) – primary(k) is pointer to the data that was augmented by symmetry to yield the kth record
  - aug (int 1-D array) – aug(k) indicates the symmetry that was used to yield the kth record, e.g., if there are n_sym total symmetries (e.g., n_sym=8 for 'full'), then aug is in [1:n_sym]. The pairs [primary(k) aug(k)] are unique; a contiguous block of n_sym pairs share one value of primary(k) and have sequential values of aug(k)
  - syms_applied (cell array of char) – syms_applied{k} are the binary texture coordinates after symmetry transformation for record k

Note regarding domain names, stimulus coordinates, and setup files

The domain name is determined from the data file name based on the string preceding '_coords' (or, opts_read.coord_string, if customized). This in turn determines how stimulus coordinates are specified.
- For non-reserved domain names (opts_read.type_class_aux, or any domain in opts_read.domain_list_def), stimulus coordinates are specified by 'type_coords' and no setup file is used.
- Other domain names are reserved, and stimulus coordinates are specified by a 'setup file', whose suggested name is constructed from the data file name up to the string preceding opts_read.coord_string, and appending opts_read.setup_suffix
- The domain name is also used to determine a 'type_class', which is used internally. This is carried out in psg_coorddata_parsename, as follows:
  - The domains 'faces*' are reserved; type_class is faces_mpi; used with a setup file
  - The domain 'irgb' is reserved; type_class is 'irgb'; used with a setup file
  - The domain 'mater' is reserved; type_class is 'mater'; used with a setup file
  - The domain opts_read.type_class_aux is NOT reserved; type class is type_class_aux; no setup file used
  - Any domain that is listed in opts_read.domain_list_def is NOT reserved; type class is 'domain'; no setup file used
- Otherwise, type_class is set to opts_read.type_class_def, ('btc' unless customized); used with a setup file
When a dataset structure is written by rs_write_coordsets, the setup file information, if any, will be embedded. When that file is read in and an embedded setup file is detected, no setup file is requested.
Also see note below regarding customization

Note regarding quadratic form model

Quadratic form models may be used to generate a coordinate structure based on stimulus coordinates.
The file must a cell array r, and each r{imodel} has at least has structure r The quadratic form model file must contain a field r, which must contain
- results.qfit (float 2-D array): a symmetric matrix indicating the quadratic form which, when applied to stimulus coordinates, yields the model coordinates
- setup.label (char): a label for the model
See demos/opposites_qform_example.mat for an example file with four quadratic form models, applicable to a stimulus set in which stimuli have three coordinates

Note regarding customization

The defaults for the following parameters can be set by editing the line containing generic.opts_read.[param_name] in rs_aux_defaults_define, running it once, and saving the workspace as rs_aux_defaults.mat.
- if_gui
- ui_filter
- if_log
- if_warn
- if_auto
- data_fullname_def
- setup_fullname_def
- domain_list_def
- setup_suffix
- permutes_ok
- coord_string
- type_class_aux
- if_symaug
- sym_apply
- if_symaug_log
The defaults for the following parameters can be set by editing the line containing generic.opts_qpred.[param_name] in rs_aux_defaults_define, running it once, and saving the workspace as rs_aux_defaults.mat.
- qform_datafile_def
- qform_modeltype

See also: rs_aux_customize, rs_findrays, rs_check_coordsets, PSG_GET_COORDSETS, PSG_COORDDATA_PARSENAME.

Source code in src/rs_get_coordsets.m

function [data_out,aux_out]=rs_get_coordsets(fullnames,aux)
% [data_out,aux_out]=rs_get_coordsets(fullnames,aux): reads one or more `coordinate files` into a `dataset structure`
% 
% This is the preferred method for bringing coordinates from one or more files into a `dataset structure` suitable for display and geometrical analysis.
%
%  - For files generated by analysis of similarity judgments via [??how to refer to Python package], this will also import the goodness-of-fit data.
%  - A `ray structure` will be created.
%  - If only a single data file is to be read, `rs_read_coorddata` can also be used.
%  - This can also create a `coordinate structure` based on a `quadratic form model`, and can augment `binary texture domain` data by symmetry.
%  - To import coordinates generated externally or that exist as arrays rather than in a file, use `rs_import_coordsets`.
%
%  - For non-interactive reading, provide aux.nsets, aux.opts_read.data_fullnames, aux.opts_read.input_type, and set aux.opts_read.if_auto=1.
%
% Args:
%   fullnames (char or cell array of char): full file name with path; if empty, it will be requested interactively. File names must contain the string '_coords'. See note below regarding setup files.
%
%   aux (struct): a structure, can be omitted, with fields 
%
%     - nsets (int): number of records to read, if zero (default), then requested at console, with console option to switch to gui
%     - opts_read (struct): can be omitted, with fields listed below
%
%        - if_gui (int): 1 to use graphical interface to get files if file names are not supplied (default), 0 to use console; ignored if nsets=0; see note below regarding customization
%        - if_log (int): 1 to log progress, 0 to omit; default is 1; see note below regarding customization
%        - if_auto (int): 1 not to ask for confirmations, 0 to ask; default is 0; see note below regarding customization
%        - nfiles_max (int): maximum number of files to read; default is 100
%        - input_type (int or int 1-D array): 1 for reading a coordinate file, 2 for reading a `quadratic form model`, 0 to ask; cycled through for each dataset; default is 0;  see note below regarding customization
%        - data_fullname_def (char): prompt for data file name; see note below regarding customization
%        - ui_filter (char): file name filter that appears in graphical user interface when if_gui=1; defaults to '*_coords';see note below regarding customization
%        - if_warn (int): 1 to show warnings, 0 to omit;  default is 1; see note below regarding customization
%
%        - **Options related to stimulus coordinates, domains, and setup files (see notes below)** 
%        - type_coords (float 2-D array): stimulus coordinates to use for non-reserved domain names, one row for each stimulus; default is empty
%        - type_coords_def (char): stimulus coordinates to use if type_coords is unspecified or empty; options are 'none', 'zeros', 'ones', or 'eye', see `rs_import_coordsets`; default is 'none'; see `rs_import_coordsets` for further details and how to customize
%        - domain_list_def (cell array): paradigm names for generic domain experiments; default is {'cars','tools','dwellings'}; determined by customization during install
%        - paradigm_type_def (char): default paradigm type; default is determined by customization during install
%        - coord_string (char): a string in a data file name that delineates the end of the domain name; defaults to '_coords'
%        - setup_fullnames (cell array): setup file full names; if empty, will be requested at console or via gui
%        - setup_fullname_def (char): prompt for setup file name
%        - setup_suffix (char): suffix added to paradigm name to create the setup file name; default is 'S'
%        - type_class_aux (char): a specified domain name and type_class
%
%        - **Options for symmetry augmentation (`binary texture domain` data only)**
%        - if_symaug (int): 1 to enable symmetry augmentation, 0 to omit, -1 to ask; default is 0
%        - if_symaug_log (int): 1 to log symmetry augmentation, 0 to omit; default is 0
%        - sym_apply (char): type of symmetry to apply; default is 'full', options are  'hflip','vflip','rot180','rot90','hvflip','diag','full','none'
%
%        - **Options for internal use and maintenance**
%        - permutes (struct): typically omitted; each field is a suggested ray permutation for the corresponding paradigm name, e.g., permutes.bgca=[2 1 3 4] specifies a reordering of the rays for paradigm 'bgca'
%        - permutes_ok (int): typically omitted; 1 to accept suggested ray permutation, 0 to keep standard order; default is 1; see note below regarding customization
%        - if_uselocal (int): typically omitted; 0 to use options in rs_aux_defaults, 1 is reserved for maintenance; default is 0
%
%     - opts_qpred (struct): a structure, can be omitted, with options for `quadratic form model`, with fields
%
%        - qform_datafile_def (char): default name of the file with quadratic form models containing a structure r; see notes below quadratic form model and customization
%        - qform_modeltype (int or int 1-D array): list of indexes into qform_datafile_def for choice of model, used cyclically across records as needed. 0 -> ask at console; detault is 1 see note below re customization
%        - if_log (int): 1 to log calculation of quadratic form model, 0 to omit; default is 0
%        - if_pca_centroid (int): 1 to force centroid of coordinates to 0 via PCA method, 0 to omit; default is 1
%        - negeig_makezero (int): 1 to set negative eigenvalues to 0, 1 to omit; default is 1
%        - negtol (float): eigenvalues between 0 and -negtol are set to zero before checking for negative eigenvalues; default is 10^-7
%
%     - opts_rays (struct): options for identifying rays, may be omitted; see `rs_findrays` for details
%
%     - opts_check (struct): options for consistency checking, with field
%
%         - if_warn (int): 1 to show warnings when datasets are checked for consistency, 0 to suppress; default is 1
%
% Returns:
%   data_out (struct): `dataset structure`, containing one records, with fields
%
%     - ds (cell array): `coordinate structure`, ds{k}{idim} is an array of [nstims idim] of coordinates for the kth record
%     - sas (cell array): `stimulus metadata structure`, sas{k} is the stimulus metadata for the kth record
%     - sets (cell array): `set metadata structure`, sets{k} is the response metadata for the kth record
%
%   aux_out (struct): auxiliary outputs and parameter values used, with fields
%
%     - warnings (char): warnings generated during consistency check
%     - warn_bad (int): number of warnings that prevent further processing
%     - rayss (cell array of struct): rayss{k} is the `ray structure` for the kth record
%     - opts_read (cell array of struct): opts_read{k} is aux.opts_read for reading the kth record, with defaults filled in
%     - opts_rays (cell array of struct): opts_rays{k} are the options used for creating the `ray structure` for the kth record
%     - opts_qpred (cell array of struct): opts_qpred{k} are the options used for creating the `quadratic form model` for the kth record along with
%
%         - var_tot (float): total variance of the coordinates
%         - var_ex (float 1-D array): variance explained by each dimension of the model
%         - predcoords (float 2-D array): predcoords(istim,idim) is the model coordinate for stimulus istim in dimension idim
%
%     - opts_check (struct): aux.opts_check, with defaults filled in
%     - syms_list (struct): structure with results of symmetry augmentation
%
%         - sym_apply (char): the symmetry applied, see 'sym_apply' above
%         - primary (int 1-D array): primary(k) is pointer to the data that was augmented by symmetry to yield the kth record
%         - aug (int 1-D array): aug(k) indicates the symmetry that was used to yield the kth record, e.g., if there are n_sym total symmetries (e.g., n_sym=8 for 'full'), then aug is in [1:n_sym]. The pairs [primary(k) aug(k)] are unique; 
%         a contiguous block of n_sym pairs share one value of primary(k) and have sequential values of aug(k)
%         - syms_applied (cell array of char): syms_applied{k} are the `binary texture` coordinates after symmetry transformation for record k
%
% Notes: Note regarding domain names, stimulus coordinates, and setup files
%     - The domain name is determined from the data file name based on the string preceding '_coords' (or, opts_read.coord_string, if customized).  This in turn determines how stimulus coordinates are specified.
%
%         - For non-reserved domain names (opts_read.type_class_aux, or any domain in opts_read.domain_list_def), stimulus coordinates are specified by 'type_coords' and no setup file is used.
%         - Other domain names are reserved, and stimulus coordinates are specified by a 'setup file', whose suggested name is constructed from the data file name up to the string preceding opts_read.coord_string, and appending opts_read.setup_suffix
%         - The domain name is also used to determine a 'type_class', which is used internally. This is carried out in psg_coorddata_parsename, as follows:
%
%             - The domains 'faces*' are reserved; type_class is faces_mpi; used with a setup file
%             - The domain 'irgb' is reserved; type_class is 'irgb'; used with a setup file
%             - The domain 'mater' is reserved; type_class is 'mater'; used with a setup file
%             - The domain opts_read.type_class_aux is NOT reserved; type class is type_class_aux; no setup file used
%             - Any domain that is listed in opts_read.domain_list_def is NOT reserved; type class is 'domain'; no setup file used
%
%         - Otherwise, type_class is set to opts_read.type_class_def, ('btc' unless customized); used with a setup file
%
%     - When a `dataset structure` is written by `rs_write_coordsets`, the setup file information, if any, will be embedded. When that file is read in and an embedded setup file is detected, no setup file is requested.
%     - Also see note below regarding customization
%
% Notes: Note regarding quadratic form model
%     - Quadratic form models may be used to generate a `coordinate structure` based on stimulus coordinates.
%     - The file must a cell array r, and each r{imodel} has at least has structure r The quadratic form model file must contain a field r, which must contain
%
%         - results.qfit (float 2-D array): a symmetric matrix indicating the quadratic form  which, when applied to stimulus coordinates, yields the model coordinates
%         - setup.label (char): a label for the model
%
%     - See demos/opposites_qform_example.mat for an example file with four quadratic form models, applicable to a stimulus set in which stimuli have three coordinates
%
% Notes: Note regarding customization
%     - The defaults for the following parameters can be set by editing the line containing generic.opts_read.[param_name] in  `rs_aux_defaults_define`, running it once, and saving the workspace as rs_aux_defaults.mat.
%
%         - if_gui
%         - ui_filter
%         - if_log
%         - if_warn
%         - if_auto
%         - data_fullname_def
%         - setup_fullname_def
%         - domain_list_def
%         - setup_suffix
%         - permutes_ok
%         - coord_string
%         - type_class_aux
%         - if_symaug
%         - sym_apply
%         - if_symaug_log
%
%     - The defaults for the following parameters can be set by editing the line containing generic.opts_qpred.[param_name] in  `rs_aux_defaults_define`, running it once, and saving the workspace as rs_aux_defaults.mat.
%
%         - qform_datafile_def
%         - qform_modeltype
%
%  See also: RS_AUX_CUSTOMIZE, RS_FINDRAYS, RS_CHECK_COORDSETS, PSG_GET_COORDSETS, PSG_COORDDATA_PARSENAME.
%
if (nargin<=1)
    aux=struct;
end
aux=filldefault(aux,'opts_read',struct);
type_coords_def=getfield(getfield(rs_aux_customize(setfield(struct(),'opts_import',struct()),'rs_import_coordsets'),'opts_import'),'type_coords_def');
aux.opts_read=filldefault(aux.opts_read,'type_coords_def',type_coords_def);
%
aux=filldefault(aux,'opts_rays',struct);
%
aux=filldefault(aux,'opts_qpred',struct);
%
aux=filldefault(aux,'nsets',0);
%
aux=filldefault(aux,'opts_check',struct);
aux.opts_check=filldefault(aux.opts_check,'if_warn',1);
%
aux=rs_aux_customize(aux,'rs_get_coordsets');
%
data_out=struct;
aux_out=struct;
aux_out.warnings=[];
aux_out.warn_bad=0;
%
if ~iscell(fullnames)
    fullnames_list{1}=fullnames;
else
    fullnames_list=fullnames;
end
nsets=abs(aux.nsets);
nsets_named=length(fullnames_list);
if isempty(fullnames) | isempty(fullnames_list)
    nsets_named=0;
end
if nsets_named==0
    %If fullnames is empty, then set nsets to psg_get_coordsets as positive or negative
    if nsets>0
        if aux.opts_read.if_gui>0
            aux.nsets=-nsets; %flag for psg_get_coordsets to use gui
        else
            aux.nsets=nsets;
        end
    end
else %If fullnames is not empty, check that its length agrees with nsets and that each contains _coords
    if isempty(nsets)
        nsets=nsets_named;
        aux.nsets=nsets;
    end       
    if nsets_named~=nsets
        wmsg=sprintf('number of files listed (%3.0f) disagrees with number of files specified (%3.0f)',nsets_named,nsets);
        aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_read.if_warn));
    end
    for iset=1:nsets_named
        if ~contains(fullnames_list{iset},aux.opts_read.coord_string)
            wmsg=sprintf('file name %2.0f (%s) does not contain the required tag ''%s''',iset,fullnames_list{iset},aux.opts_read.coord_string);
            aux_out=rs_warning(wmsg,1,setfield(aux_out,'if_warn',aux.opts_read.if_warn));
        end
    end
    %attempt to read automatically
    aux.opts_read.data_fullnames=fullnames_list;
    %create setup files
    aux.opts_read.setup_fullnames=cell(1,nsets_named);
    for iset=1:nsets_named
        parsed=psg_coorddata_parsename(fullnames_list{iset},aux.opts_read);
        aux.opts_read.setup_fullnames{iset}=parsed.setup_fullname_def;
    end
end
if aux_out.warn_bad==0
    [sets,ds,sas,rayss,opts_read_used,opts_rays_used,opts_qpred_used,syms_list]=...
        psg_get_coordsets(aux.opts_read,aux.opts_rays,aux.opts_qpred,aux.nsets);
    %find rays via rs_findrays, so that coords are sought in type_coords or btc_augcoords
    for iset=1:nsets
        if strcmp(opts_read_used{iset}.type_class,'domain')
            [rayss{iset},wmsg,opts_rays_used{iset}]=rs_findrays(sas{iset},opts_read_used{iset}.setup_fullname,aux.opts_rays);
            if ~isempty(wmsg)
                aux_out=rs_warning(wmsg,0,aux_out);
            end
        end
    end
    %
    data_out.sets=sets;
    data_out.ds=ds;
    data_out.sas=sas;
    %
    aux_out.opts_check=aux.opts_check;
    aux_out.opts_read=opts_read_used;
    aux_out.opts_rays=opts_rays_used;
    aux_out.opts_qpred=opts_qpred_used;
    aux_out.rayss=rayss;
    aux_out.syms_list=syms_list;
    %
    %check internal consistency
    %
    for iset=1:nsets
        data_check=struct;
        data_check.ds{1}=data_out.ds{iset};
        data_check.sas{1}=data_out.sas{iset};
        data_check.sets{1}=data_out.sets{iset};
        check=rs_check_coordsets(data_check,setfield(aux.opts_check,'set_num_offset',iset-1));
        if ~isempty(check.warnings) %since strvcat([],[])~=[]
            aux_out.warnings=strvcat(aux_out.warnings,check.warnings);
            warn_leadin=getfield(getfield(rs_aux_customize(struct()),'overall'),'warn_leadin');
            for k=1:size(check.warnings,1)
                disp(cat(2,warn_leadin,check.warnings(k,:)));
            end
        end
        aux_out.warn_bad=aux_out.warn_bad+check.warn_bad;
    end
    %
else
    disp('cannot proceed');
    warn_leadin=getfield(getfield(rs_aux_customize(struct()),'overall'),'warn_leadin');
    for k=1:size(aux_out.warnings,1)
        disp(cat(2,warn_leadin,aux_out.warnings(k,:)));
    end
end
return
end