Search code examples
matlabimportcovariance

reading a covariance matrix from a FACSIMILE log file into MATLAB


I am trying to get a co-variance matrix out of a FACSIMILE logfile (this is just a generic textfile with a .log extension), as well as a string array or similar listing the parameter names, in the order that the file has put them in.

I've included the relevant section of a log file. In this one it occurs at line 3285, though they tend to move around - and both the covariance matrix and the block in which parameter names are listed can be identified by their headers!

FITTED VALUES, ACCURACIES AND CONFIDENCE LIMITS


NO.   NAME                       VALUE       SDLN   5 PERCENT  95 PERCENT
   1 KA1                        5.2353E-04  0.1809  3.8880E-04  7.0496E-04
   2 KA2                        6.7624E-02  0.3834  3.5994E-02  1.2705E-01
   3 KD1                        3.5626E-05  0.9008  8.0949E-06  1.5679E-04
   4 KD2                        1.3444E-01  0.3023  8.1764E-02  2.2104E-01
   5 KAA1                       4.7226E-04  1.8292  2.3302E-05  9.5715E-03
   6 KAA2                       1.7067E+04  1.7779  9.1613E+02  3.1793E+05
   7 KAAAA                      1.8234E+02  0.3205  1.0763E+02  3.0893E+02
   8 R1                         1.3568E+00  0.1278  1.0996E+00  1.6743E+00
   9 R2                         2.9313E-01  0.0268  2.8048E-01  3.0635E-01
  10 R3                         1.3338E-01  0.0303  1.2689E-01  1.4021E-01
  11 T1B                        9.9068E-01  0.2833  6.2163E-01  1.5788E+00
  12 T2B                        3.4526E-01  0.2362  2.3412E-01  5.0918E-01
  13 KM11                       3.5125E-04  0.1825  2.6015E-04  4.7425E-04
  14 KM21                       3.5625E-04  0.1881  2.6144E-04  4.8544E-04
  15 KM31                       1.5056E-04  0.1899  1.1016E-04  2.0576E-04
  16 CTRANSFORM                 1.9530E-01  0.2302  1.3374E-01  2.8518E-01

  CORRELATION MATRIX COMPONENTS

 COLUMN     1      2      3      4      5      6      7      8      9     10
           11     12     13     14     15     16

 ROW    1  1.000  0.277  0.048  0.336 -0.360  0.444  0.310 -0.767 -0.448 -0.471
          -0.784 -0.484 -0.761 -0.875 -0.882  0.782

 ROW    2  0.277  1.000  0.235  0.910 -0.136  0.478  0.222  0.140 -0.144 -0.294
           0.186 -0.275  0.255  0.054  0.001 -0.229

 ROW    3  0.048  0.235  1.000  0.338 -0.021  0.122  0.416  0.109  0.565  0.144
           0.073 -0.366  0.152  0.083  0.107 -0.129

 ROW    4  0.336  0.910  0.338  1.000 -0.306  0.364  0.554  0.242 -0.269 -0.137
           0.237 -0.065  0.315  0.103  0.036 -0.283

 ROW    5 -0.360 -0.136 -0.021 -0.306  1.000 -0.454 -0.325  0.035  0.547  0.404
           0.064 -0.271  0.058  0.122  0.105 -0.066

 ROW    6  0.444  0.478  0.122  0.364 -0.454  1.000 -0.119 -0.252 -0.329 -0.943
          -0.189 -0.254 -0.212 -0.340 -0.238  0.218

 ROW    7  0.310  0.222  0.416  0.554 -0.325 -0.119  1.000  0.150 -0.156  0.301
           0.041  0.124  0.158  0.065  0.044 -0.116

 ROW    8 -0.767  0.140  0.109  0.242  0.035 -0.252  0.150  1.000  0.088  0.408
           0.990  0.676  0.979  0.964  0.935 -0.984

 ROW    9 -0.448 -0.144  0.565 -0.269  0.547 -0.329 -0.156  0.088  1.000  0.424
           0.094 -0.480  0.157  0.245  0.262 -0.154

 ROW   10 -0.471 -0.294  0.144 -0.137  0.404 -0.943  0.301  0.408  0.424  1.000
           0.336  0.236  0.388  0.473  0.364 -0.384

 ROW   11 -0.784  0.186  0.073  0.237  0.064 -0.189  0.041  0.990  0.094  0.336
           1.000  0.639  0.979  0.959  0.933 -0.986

 ROW   12 -0.484 -0.275 -0.366 -0.065 -0.271 -0.254  0.124  0.676 -0.480  0.236
           0.639  1.000  0.573  0.604  0.591 -0.592

 ROW   13 -0.761  0.255  0.152  0.315  0.058 -0.212  0.158  0.979  0.157  0.388
           0.979  0.573  1.000  0.971  0.944 -0.998

 ROW   14 -0.875  0.054  0.083  0.103  0.122 -0.340  0.065  0.964  0.245  0.473
           0.959  0.604  0.971  1.000  0.977 -0.977

 ROW   15 -0.882  0.001  0.107  0.036  0.105 -0.238  0.044  0.935  0.262  0.364
           0.933  0.591  0.944  0.977  1.000 -0.948

 ROW   16  0.782 -0.229 -0.129 -0.283 -0.066  0.218 -0.116 -0.984 -0.154 -0.384
          -0.986 -0.592 -0.998 -0.977 -0.948  1.000

If you can help everyone who uses FACSIMILE will be grateful!


Solution

  • I've written some code that gets you half way there! This reads in the parameter estimates from the table above the co-variance matrix.

    function out = GET_parameter_estimates_from_LogFile(FileAddress)
    
    out                              = cell(0);
    
    %sometimes its a cell
    if iscell(FileAddress)
        FileAddress=FileAddress{1};
    end
    
    %open it!
    fid = fopen(FileAddress,'r');
    
    if fid<0 %the file doesnt exist
        display(['the file ',FileAddress,' is missing'])
        out={};
    else %the file exists
    
    TableNumber=0
    while ~feof(fid)
            %there are multiple tables of parameter values in the file and we want
        %parameter values from all of them!
        TableNumber=TableNumber+1;
        % Find the line the (next) table starts on!
        while ~feof(fid)
            line = fgetl(fid);
            %is the 5 percent and 95 percent enough of a marker to identify the
            %header line?
            if ~isempty(strfind(line,'PARAMETER NUMBER,')) && ~isempty(strfind(line,'TYPE,'))
                break %found a part of the logfile where parameter values are listed
            end
        end 
    
        if exist('line','var')
        line(strfind(line,'NUMBER')-1)='_';
        else
            display('cant find the SDLN table in ', FileAddress,'using Dummy logfile') 
        end
    
        if length(line)>1 % don't do anything if its the last line of the file which is length 1
            header                             = textscan(line,'%s','delimiter',' ');
            header                             = header{1};
            unwanted_elems                     = zeros(0);
            for i=1:length(header)
                if isempty(header{i})
                    unwanted_elems(end+1)=i;
                end
            end
            if ~isempty(unwanted_elems)
                header(unwanted_elems)=[];
            end
            if size(header,1) > size(header,2)
                header = header';
            end
            out(1,:,TableNumber)=header;
            k=2;
    
            % get  the stuff under the header
            while ~feof(fid)
                line = fgetl(fid);
                %first non-table line is just a single space
                if length(line)<=1
                    break
                end
                header = textscan(line,'%s','delimiter',' ');
                header = header{1};
                unwanted_elems = zeros(0);
                for i=1:length(header)
                    if isempty(header{i})
                        unwanted_elems(end+1)=i;
                    end
                end
                if ~isempty(unwanted_elems)
                    header(unwanted_elems)=[];
                end
                if size(header,1) > size(header,2)
                    header = header';
                end
                out(k,:,TableNumber)=header;
                k=k+1;
            end
        end
    end
        fclose(fid);
    end
    
    %out is the output
    end