Search code examples
c++csvfile-ioprocessing-efficiency

Writing Floats to a CSV File Using C++ Efficiently

I am currently using Microsoft Kinect C++ to get skeletal coordinate data (x,y,z) along with a time_stamp. So this translates to 61 columns of data per write.

What I am currently doing is setting up the csv file w/ the column headers at the beginning of the main() method. Then I run an infinite while loop to process color/depth frame skeleton data and save the skeleton data into this csv file: pseudo (cuz my code is too long): main method() { ostream myfile; myfile.open(cvsfile); setup myfile stuff;

...
while(1) {
    get color frame ready;
    get depth frame ready;
    get skelframe() ready;

    ... other stuff that isn't related to the problem
}
end of main method} 
... 
skelframe() {
    setsup skeleton frame and image;

    ...
    get the 60 data points (x,y,z) of 20 joints;
    ostream myfile;
    myfile.open(csvfile)
    myfile<<all 60 data points + time_stamp<<endl;
}

So in the skelframe() method, I am setting up the skeleton and then obtaining the x,y,z coordinates. I have a couple of questions:

  1. Should I pass in myfile to the skelframe(), so that I don't have to do the setup for the csv file every time the method loops through the while(1)? If so, do I just pass it as skelframe(..., myfile), or is there some other fancy way of doing this?

  2. When I am writing all those 60 datapoints into the csv file, is there an efficient way to do this using a buffer of some sorts? I thought of using a fixed array size of 60 floats+1 string with ","s and then writing that entire buffer to the csv file; looked into stringstream, but no idea how to implement. Need some help w/ example code cuz pretty lost and noob at this lower level coding concepts.

Thanks!

Solution

  • Writing to myfile was not the bottleneck, but I switched the code around, so that the file remained open at all times until my program was done running.

    The real bottleneck was the fact that I was saving images. I changed the image types from ".png" to ".bmp", which increased the size of the images, but increased the speed of saving. I also changed the data collection stream of skeleton from cv_8UC3 to CV_8UC1 which decreased the amount of sensors in the Microsoft Kinect my skeletal data was gathering from (it really only needs 1 data stream).

    Thanks for the help.