How to append in .mat file row or columnwise cellmatrix

I am running a simulation where i generate huge 2d sparse matrices and hence i use FIND function to only store nonzero values with their indices.

Now for each iteration of for loop i generate such matrix and because they are all of different length I use cell-array to store these configurations. But for large simulations even squeezed-off format of cell-array crosses its limits of memory and hence i want to write these cell-array while running the code i.e. for each iteration append a new element into existing mat file.

for e.g.

for n=1:10  
A=rand(5);  
[i,j,vals]=find(A);  
data={[i,j,vals]};  
save('data','data','-append');  
end

Here my final goal is to have a mat file where number of elements in "data" are 10. but i because of memory issues i cant save it outside the for loop i want to generate data{n} and append it in columnwise growing fashion. eventually giving me data{10}.

I tried to use MATFILE but it gives me error that it doesnt work with {} hence not working with cell arrays.

Thank you, Nitin

Solution

You can use matfile with cells as long as you are not trying to index into the actual cells. Remember that cells are still arrays, so you can access each cell using array indexing. For example:

>> x = {'this', 'is', 'an', 'example'};
>> x{4}

ans =

example

>> x(4)

ans = 

    'example'

The following just initializes the data. Make sure to save it with the '-v7.3' tag, so that it supports efficient partial saving and loading.

data = {};
save('data.mat', 'data', '-v7.3');

Now you can access your data with the matfile

mf = matfile('data.mat', 'Writable', true);
for n=1:10  
   A=rand(5);  
   [i,j,vals]=find(A);  
   data={[i,j,vals]};
   mf.data(end+1, 1) = data;
end

Reference: matfile documentation