I currently pulling data from a CSV file. The CSV file has ~ 89 columns and 2000 rows worth of data. I am getting several specific columns of data such as all of col:1,2,21,22,66,67 using a variety of getlines and loops. I then store that data into vectors within the loops. Once I have read through the entire file I now have 6 vectors full of data that I want. I make some adjustments to that data and store it back into a vector. I now want to place that new data back into those columns I took them out of without actually picking up/out the other data that I don't want. What would be the best approach for this? As I don't want to make 89 vars to hold all that other data I would much rather write over those columns in particular or something similar.
As I don't want to make 89 vars to hold all that other data I would much rather write over those columns in particular or something similar.
Instead of using 6 vectors to store column data, you can use one vector of strings to hold the data from one row. Then you update the elements at 1,2,21,22,66,67 in that vector and write it to another file.
std::vector<std::string> row; // 89 elements after read and parse a row.
Processing 500,000 rows this way should be fast enough. If it is not, try a column-oriented database, e.g. OpenTSDB