How can I save a matrix which grows in time?

Question

I have a 3D matrix whose third dimension is time, and it grows as time goes on. One way to save this on file in real-time would be to use this format:

# Created by Octave 6.3.0, Wed Sep 01 10:30:13 2021 BST <pietrom@UNKNOWN>
# name: A
# type: matrix
# ndims: 3
 3 4 2
 1
 5
 9
 2
 6
 10
 3
 7
 11
 4
 8
 12
 13
 17
 21
 14
 18
 22
 15
 19
 23
 16
 20
 24

where the time dimension is the third one. Here I show 2 time steps.

To add another time step, I should update the line where the matrix size is specified, i.e. going from "3 4 2" to "3 4 3", and append the new data at the end of the file.

However this approch is not optimal. Is there a (text based) Octave/MATLAB file format which does not require to update the header for every insertion?

Answer 1

Since you already know the size of first two dimensions, you can simply append the new time step (or time steps block) as a column-, or better for readability, row-vector.

A = rand(3, 4, 100);
Nblock = 5;  % number of time steps to be written as a block
fmt = [repmat('%.4f \t', [1, size(A, 1)*size(A, 2)]), '\n']; % or whatever format you want to write your data in

fid = fopen('test.txt', 'a');
fprintf(fid, 'Your header');
for i = 1:size(A, 3)/Nblock
    idx = (i - 1)*Nblock;
    for j = 1:Nblock
        fprintf(fid, fmt, reshape(A(:, :, idx + j), [1, size(A, 1)*size(A, 2)]));
    end
end
fclose(fid);

Note that this is written in Matlab, and some commands might change in Octave.