Search code examples
pythonpython-3.xcolumnsorting

I'm trying to sort/split a csv in python


I got a .csv file with 18k lines of data from 11 different measuring devices. I'm trying to copy/write a file for each measuring devices so I can plot them later, get averages easier. However, with this code, I put together, scrambled together from YT tutorials and web sources the only thing that's being written in these files are the "fieldnames"/the names of the columns(whatever the right name for those things are. It just stops after inserting the first line of the .csv instead of looking for the right value in each line and inserting it into the new .csv files

I've tried to use a for loop which has 11 different if/elif conditions in it which I thought would filter the column of the device_id to the right device file.

import csv
with open('Data.csv', 'r') as Data_puntenOG:
    Data_punten = csv.DictReader(Data_puntenOG)
    for line in Data_punten:
        if line['device_id'] == 'prototype01':
            with open('HS361.csv', 'w') as HS361:        
                csv_HS361 = csv.writer(HS361)
                csv_HS361.writerow(line)
        elif line['device_id'] == "prototype02":
            with open('MinID8.csv', 'w') as MinID8:
                csv_MinID8 = csv.writer(MinID8)
                csv_MinID8.writerow(line)

and then 9 more of the same elif lines with different names/ conditions from prototype03 until prototype12 with the exception of 9, because that one was not in the .csv file

11 files with only the first line of the .csv (id,device_id,measurement_type,measurement_value,timestamp) instead of a large pile of lines with data from the .csv file


Solution

  • if you have installed pandas, this will read the file and write out all rows with the same 'device_id' to a separate file with the name of the file being the 'device_id'.

    import pandas as pd
    
    df = pd.read_csv('Data.csv')
    
    EDIT:
    for id in df['device_id'].unique():
        df[df['device_id'] == id].to_csv(f"{id}.csv")