Improving space complexity while reading from a file

Question

I have an arbitrarily long line of integers (or floating point values) separated by commas in a file:

1,2,3,4,5,6,7,8,2,3,4,5,6,7,8,9,3,...  (can go upto >100 MB)

Now, I have to read these values and store them in an array.

My current implementation looks like this:

 float* read_line(int dimension)
   {
     float *values = new float[dimension*dimension]; // a line will have dimension^2 values
     std::string line;
     char *token = NULL, *buffer = NULL, *tmp = NULL;
     int count = 0;

     getline(file, line);
     buffer = new char[line.length() + 1];
     strcpy(buffer, line.c_str());
     for( token = strtok(buffer, ","); token != NULL; token = strtok(NULL, ","), count++ )
       {
         values[count] = strtod(token, &tmp);
       }
     delete buffer;
     return values;
   }

I don't like this implementation because:

• Using ifstream the entire file is being loaded into the memory, and then being cloned into a float []
• There is unnecessary duplication ( conversion from std::string to const char*)

What are ways to optimize memory utilization?

Thanks!

Answer 1

Something like this?

float val;
while (file >> val)
{
  values[count++] = val;
  char comma;
  file >> comma; // skip comma
}