Writing and loading a large array to/from a file in C++

I'm creating a large 2D array(5000*5000) of data which I need to use later in my program. The process of filling up the array takes 10 seconds but the array data in the end are always the same. So, I was thinking I could avoid these 10 seconds by reading the array from a file each time I run my program.

However, I can't think of an efficient way to write my array into a file. The array consists of floats and writing all the floats with a space between or even a line, I could easily read them later and recreate the array. This obviously turned out to be a bad idea as I end up with a huge txt file of million lines. Reading from that takes even longer and the file takes up way too much space.

How does someone save an array into a file so it can be loaded later?

Edit: You suggested me to try saving in binary format. The tutorials I've found have confused me a bit. Here's my effort:

//2d array[size][size]
vec4** F = new vec4*[size];
for (int i = 0; i < size; i++) {
    F[i] = new vec4[size];
}
// Array is filling up, I won't include this part
//...........
// Array is ready.

//Trying to write the array in a file.
FILE* pFile;
pFile = fopen("myfile.bin", "wb");
fwrite(F, sizeof(vec4*), sizeof(F), pFile);

//Other method
std::ofstream out("filename.data", std::ios_base::binary);
out.write((char*)F, sizeof(vec4)*(size*size));

Both these methods create empty files. Maybe the fact that it's a 2d array complicates fwrite and write?

Solution

Simply write out the entire array as-is in its original binary form, not in a textual form.

vec4* F = new vec4[size*size];

// fill and use the array as needed...

std::ofstream out("filename.data", std::ios_base::binary);
out.write((char*)F, sizeof(vec4)*(size*size));

delete[] F;

You can then read the file back into your array as-is:

vec4* F = new vec4[size*size];

std::ifstream in("filename.data", std::ios_base::binary);
if (!in.read((char*)F, sizeof(vec4)*(size*size)))
{
    // generate new values as needed...
}

// use the array as needed...

delete[] F;

Alternatively, if you use a memory-mapped file (CreateFileMapping()/MapViewOfFile() on Windows, mmap() on Linux, etc), then you don't even have to allocate an array and read the file into it, you can just access the file data directly using a memory-mapped data pointer, eg:

vec4 *F = map the file ...; // <-- use platform-specific APIs for this!
bool mapped = (F != nullptr);
if (!mapped)
{
    F = new vec4[size*size];

    // generate new values as needed...

    std::ofstream out("filename.data", std::ios_base::binary);
    out.write((char*)F, sizeof(vec4)*(size*size));
}

// use the array as needed ...

if (mapped)
    unmap the file; // <-- use platform-specific APIs for this...
else
    delete[] F;

UPDATE: if you want to use a 2-dimensional sparse array, you will have to account for that a little differently in your file I/O, eg:

vec4** F = new vec4*[size];
for (int i = 0; i < size; ++i) {
    F[i] = new vec4[size];
}

{
    std::ifstream in("filename.data", std::ios_base::binary);
    if (in)
    {
        for (int i = 0; i < size; ++i) {
            in.read((char*)F[i], sizeof(vec4)*size);
        }
    }

    if (!in)
    {
        // generate new values as needed...
    }
}

// use the array as needed...

{
    std::ofstream out("filename.data", std::ios_base::binary);
    for (int i = 0; i < size; ++i) {
        out.write((char*)F[i], sizeof(vec4)*size);
    }
}

for (int i = 0; i < size; ++i) {
    delete[] F[i];
}
delete[] F;