Data get lost when written to binary file - C++

Question

I have an array including 4994 vectors. I want to save the array in a file in binary format. Since I'm not used to doing that, I did a test to see if the array is successfully copied to the file. So, I first save the array in a file and then load the array from that file. Comparing the data, I find that they are different after a specific point :

vec3* points = new vec3[4994];
vec3* pointsRead = new vec3[4994];

//
//Adding data in 'points'...
//

//Write 'points' in binary file
std::ofstream suzannePointsFile("suzannePoints.data", std::ios_base::binary);
suzannePointsFile.write((char*)points, sizeof(vec3) * 4994);

//Read 'points' into 'pointsRead'
std::ifstream suzanneRead("suzannePoints.data", std::ios_base::binary);
suzanneRead.read((char*)pointsRead, sizeof(vec3) * 4994);

for (int i = 0; i < 4994; i++) {
    if (points[i] != pointsRead[i]) {
        //When debugging, the if statement is true when i>=4778.
        //Meaning that the data is different from this point on.
    }
}

I figured there is something wrong with the write or read functions. So I tried replacing their second argument (sizeof(vec3)*4994) for some random greater numbers and this is what happened:

• When replacing 4994 with 5100, I have exactly the same problem. The data is different after i=4778.
• When replacing 4994 with 5200, all the data is successfully transferred.

This makes no sense to me. Any ideas?

Answer 1

Operations on files using the C++ Standard Library are buffered. The reason is that writing exactly as many bytes as the programmer asks can be very inefficient. For example, a spinning hard drive or an SSD drive cannot write single bytes, but only blocks of say 4096 bytes. So if you wanted to write a single byte, you'd have to read the 4096 byte block containing it, change the one byte you wanted to write, and write the whole 4096 byte block.

If you write a whole megabyte, byte by byte, you can see that every single byte requires to read one 4096 byte block and then write such a block. To get around this, the C++ standard library collects data, and when enough data is collected, then it writes the data. And when the file is closed, then the rest of the data is written.

You called .write to write x bytes. The result is that less than x bytes were actually written, and the rest would have been written if you wrote another vector, or if you closed the file. You didn't do that, then opened another stream for the same file. The problem is that your x bytes hadn't been written yet, so they cannot be read.