Search code examples
c++boostsavebinaryfilesboost-multi-array

How to save a boost::multi_array to a file


I'm looking for a simple way to save to and load from a file a 3D boost::multi_array. As far as I know there is no methods in the Boost library which I find odd.

I don't need the file to be human readable, so a binary file would be better for performance. Can anyone point me to a ready-made solution or give me ideas on how to implement such read/write methods ?

I use 3D multi_arrays of types bool and ints so I'll need to use a template.


Solution

  • It is unnecessary to use some special serialization libraries, because your data is already serialized in the memory and you can get it as follows (suppose A is the array containing int data:

    int *data = A.data();
    size_t size = A.num_elements();
    

    You can just write it one by one to a file. A better way may be using mmap, like follows:

    int fd = open("myfile", O_RDWR);
    size_t bytes = size * sizeof(int);
    ftruncate(fd, bytes);
    void *buf = mmap(NULL, bytes, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
    memcpy(buf, data, bytes);
    close(fd);
    munmap(buf, bytes);
    

    It is very easy to reload the data from file to A. Just ommit the invocation of ftruncate and call memcpy(data, buf, bytes);.

    An even better way is that, if your data is huge, you simply store the data in a file, use mmap to map it to a memory address, then pass the address to multi_array_ref. In this case you don't need to specifically write it to file. It is aromatically done by the OS.

    The above code is considered for Linux, but I believe other platforms should have similar functionality. Error checking is omitted for clarity.