I have a list of dictionaries and I would like to convert it to dataframe. The list has the following structure:
[{'year': 2010, 'month': 1, 'weather': DataFrame},
{'year': 2010, 'month': 2, 'weather': DataFrame},
.
.
.
{'year': 2011, 'month': 12, 'weather': DataFrame}]
I would like the dataframe to have the following columns: year, month and the columns of DataFrame
Due to the fact that you did not provide an example output or just the Weather Dataframe itself, I can only suggest the following solutions.
The first one is the simplest:
import pandas as pd
your_name_list = [
{'year': 2010, 'month': 1, 'weather': DataFrame},
{'year': 2010, 'month': 2, 'weather': DataFrame}
]
result = pd.DataFrame(your_name_list)
Output:
year month weather
0 2010 1 temperature wind 0 30 0
1 2010 2 temperature wind 0 30 0
The second solution is where we need to get all the columns of the dictionary and at the same time the Weather Dataframe, provided that the weather has one line, otherwise everything will go down:
import pandas as pd
your_name_list = [
{'year': 2010, 'month': 1, 'weather': pd.DataFrame([{'temperature': 30, 'wind': 0}])},
{'year': 2010, 'month': 2, 'weather': pd.DataFrame([{'temperature': 40, 'wind': 1}])}
]
df1 = pd.DataFrame(your_name_list)
df2 = pd.DataFrame(columns=['temperature', 'wind'])
del df1["weather"]
for i in your_name_list:
df2 = df2.append(i["weather"], ignore_index=True)
result = pd.concat([df1, df2], axis=1)
Output:
year month temperature wind
0 2010 1 30 0
1 2010 2 40 1
And the last solution, which seems to me the most logical, is the weather dataframe code has only two columns (weather condition and the value itself) and we need to get from this columns and connect the result:
import pandas as pd
your_name_list = [
{'year': 2010, 'month': 1, 'weather': pd.DataFrame([{'weather_condition': "temperature", 'value': 30},
{'weather_condition': "wind", 'value': 0}])},
{'year': 2010, 'month': 2, 'weather': pd.DataFrame([{'weather_condition': "temperature", 'value': 40},
{'weather_condition': "wind", 'value': 1}])}
]
df1 = pd.DataFrame(your_name_list)
df2 = pd.DataFrame(columns=['temperature', 'wind'])
del df1["weather"]
for i in your_name_list:
df2 = df2.append(i["weather"].assign(idx=i["weather"].groupby("weather_condition").cumcount()).pivot_table(
columns="weather_condition",
values="value",
aggfunc="first"), ignore_index=True)
result = pd.concat([df1, df2], axis=1)
Output:
year month temperature wind
0 2010 1 30 0
1 2010 2 40 1
Of course, there are many nuances that I could not take into account, but I hope I could at least partially help in solving your problem.
I will try to answer your questions in the comments if you give a more detailed question.
Here's another useful link - a question similar to yours.