How to write "Big Data" to a text file using Matlab

Question

I am getting some readings off an accelerometer connected to an Arduino which is in turn connected to MATLAB through serial communication. I would like to write the readings into a text file. A 10 second reading will write around 1000 entries that make the text file size around 1 kbyte.

I will be using the following code:

%%%%%// Communication %%%%%
arduino=serial('COM6','BaudRate',9600);
fopen(arduino);
fileID = fopen('Readings.txt','w');

%%%%%// Reading from Serial %%%%%
for i=1:Samples
    scan = fscanf(arduino,'%f');
    if isfloat(scan),
        vib = [vib;scan];
        fprintf(fileID,'%0.3f\r\n',scan);
    end
end

Any suggestions on improving this code ? Will this have a time or Size limit? This code is to be run for 3 days.

Answer 1

1. Do not use text files, use binary files. 42718123229.123123 is 18 bytes in ASCII, 4 bytes in a binary file. Don't waste space unnecessarily. If your data is going to be used later in MATLAB, then I just suggest you save in .mat files

2. Do not use a single file! Choose a reasonable file size (e.g. 100Mb) and make sure that when you get to that many amount of data you switch to another file. You could do this by e.g. saving a file per hour. This way you minimize the possible errors that may happen if the software crashes 2 minutes before finishing.