Search code examples
pythonc++csvsetw

Is it possible to read files generated with C++ std::setw using Python csv reader?


I have a data file generated using C++ std::setw e.g.

file << std::scientific << std::setprecision(data_precision);  

for (double data : a_data)                                                                                        
   {                                                                                                                 
       file << std::setw(data_width) << data;                                                                    
   }

file << "\n";

Is it possible to read the data using python csv.reader or similar? I have tried the following:

with data as csvfile:
    fieldreader = csv.reader(csvfile) 
    next(fieldreader)                                                                                                                                                                    
    for row in fieldreader:                                                                  
       values.append(float(row[0]))                                                                              

which outputs the entire first row, indicating the whole row is stored as one entry. I have also tried a few different delimiters e.g. \t which didn't help.

Example output below:

#          z        phi               phi1          Massless 
 -16.0000000  0.0000000   9.9901854997e-01  1.0910677716e-19
 -16.0000000  0.0245437   9.9871759471e-01  1.6545142956e-05
 -16.0000000  0.0490874   9.9781493216e-01  3.3051500271e-05
 -16.0000000  0.0736311   9.9631097893e-01  4.9477653557e-05
 -16.0000000  0.0981748   9.9420658732e-01  6.5784269579e-05 

Solution

  • The csvfile argument to the csv.reader initializer "can be any object which supports the iterator protocol and returns a string each time its next() method is called".

    This means you could read the file by defining a generator function like the one shown below to preprocess the lines of the file to make them acceptable to the csv.reader:

    import csv
    
    def preprocess(file):
        for line in file:
            yield ','.join(line.split())
    
    values = []
    with open('cppfile.txt') as file:
        fieldreader = csv.reader(preprocess(file))
        next(fieldreader)
        for row in fieldreader:
            print(f'row={row}')
            values.append(float(row[0]))
    
    print()
    print(values)
    

    Output:

    row=['-16.0000000', '0.0000000', '9.9901854997e-01', '1.0910677716e-19']
    row=['-16.0000000', '0.0245437', '9.9871759471e-01', '1.6545142956e-05']
    row=['-16.0000000', '0.0490874', '9.9781493216e-01', '3.3051500271e-05']
    row=['-16.0000000', '0.0736311', '9.9631097893e-01', '4.9477653557e-05']
    row=['-16.0000000', '0.0981748', '9.9420658732e-01', '6.5784269579e-05']
    
    [-16.0, -16.0, -16.0, -16.0, -16.0]