I currently have the following method for writing vector<int>
objects to a binary file.
void save_config(std::string fname, std::vector<int> config) {
std::ofstream out(fname);
for (auto&& item : config)
out << item;
}
The data I need to save, however, is on the order of 60 MB and takes about 5 seconds to write with this function. Furthermore, I have to write a binary file for each iteration of an algorithm I am running, and a single iteration for an input size which generates data on this order of magnitude is probably about 500 milliseconds.
I can mask the write time behind the algorithm's execution time but, with this difference in runtime, it won't really matter. Is there any way to improve my save_config
function? Also, I'm using a binary file because I've ready that it is the fastest format to write to; but the specific format does not matter so, if someone has an alternate suggestion, I would be happy to hear it.
Whether or not it will result in a significantly faster operation will have to be tested, but the following use of the write()
function avoids iterating through the vector:
#include <fstream>
#include <vector>
void save_config(std::string fname, std::vector<int> config)
{
std::ofstream out(fname, std::ios_base::binary);
uint64_t size = config.size();
out.write(reinterpret_cast<char*>(&size), sizeof(size));
out.write(reinterpret_cast<char*>(config.data()), size * sizeof(int));
}
Note that I have also included a 'prefix' to record the size of the vector, so that this can be determined when later reading the data from the file; I have used a fixed-size type (64-bits) for this to avoid possible issues with platforms that have a 32-bit size_t
type (you should perhaps consider using a fixed-size integer type, such as int32_t
, to avoid similar issues).