I have data like (simplified for the sake of example):
list = [
{
age: 1234,
val: 0.5,
val2: 0.2
},
{
age: 1234,
val: 0.2,
val2: 0.8
},
]
I create pandas dataframe by frame = pandas.DataFrame(list) and it creates unnamed index from 0 to len(list) - 1.
Frame looks like:
age val val2
0 1234 0.5 0.2
1 1234 0.2 0.8
Then I save it to csv by frame.to_csv('file.csv') - it goes ok.
But then, I want to create another frame exactly like this, load previous frame from csv file and add them together, so new data comes after old. I don't care about the index too, preferably it could go from 0 to new length with added data.
I tried doing it by pd.concat([old_frame, new_frame], ignore_index=True) but final data has now 2 columns with index values like this:
age val val2
0 0 1234 0.5 0.2
1 1 1234 0.2 0.8
How to properly concat the frames without creating additional index column each time?
You need to change this part of code frame.to_csv('file.csv', index=False)
This is how it will look
import pandas as pd
my_list = [
{
"age": 1234,
"val": 0.5,
"val2": 0.2
},
{
"age": 1234,
"val": 0.2,
"val2": 0.8
},
]
frame = pd.DataFrame(list)
frame.to_csv('file.csv', index=False) # <- change here
old_frame = pd.read_csv('file.csv')
new_frame = pd.DataFrame(list)
combined_frame = pd.concat([old_frame, new_frame], ignore_index=True)
The result will be:
age val val2
0 1234 0.5 0.2
1 1234 0.2 0.8
2 1234 0.5 0.2
3 1234 0.2 0.8