Parsing a CSV file and read a specific column

Question

I have a CSV file that is the output of a hydrological model. I need to read a specific column of this file but because of complex structure, pandas.read_csv cannot do that. I would be grateful if somebody can help me (For example, I need to read the Rain column values).

Please download the file from here

Answer 1

The file is not a standard csv format. The program that output's this might have a better way to export the file. But if not and this is your only option you can just parse the file manually like this:

rain_values = []

with open('AnnAGNPS_TBL_Gaging_Station_Data_Hyd.csv','r') as f:
    # skip first lines until the data begins
    while not next(f).startswith('Day'): pass

    # read lines of data
    for line in f:
        try:
            # append index 6 which is the data you want from the Rain column
            rain_values.append(line.split(',')[6].strip())
        except IndexError:
            # if we got this error it's because we've reached the end of the data section
            break

print(rain_values)

If you want to have a pandas dataframe with the csv data you can do something like this to load only the data you want:

import pandas
with open('AnnAGNPS_TBL_Gaging_Station_Data_Hyd.csv','r') as f:
    # skip first lines until the data begins
    while not next(f).startswith('Day'): pass
    lines = []
    for line in f:
        if line == '\n': break
        lines.append(line.split(','))
    pandas.DataFrame(lines)