I have this massive list of dictionaries which consists of different keys in each list except for name. The key name exists in each collection but other keys are different. For example:
items = [{
'name': 'A',
'P1 v P2': 0.2
}, {
'name': 'B',
'P1 v P2': 0.3
}, {
'name': 'C',
'P1 v P2': 0.3
}, {
'name': 'D',
'P1 v P2': 0.3
}, {
'name': 'E',
'P1 v P2': 0.3
}, {
'name': 'A',
'P2 v P3': 0.04
}, {
'name': 'B',
'P2 v P3': 0.9
}, {
'name': 'C',
'P2 v P3': 0.2
}, {
'name': 'D',
'P2 v P3': 0.3
}, {
'name': 'E',
'P2 v P3': 0.5
}, {
'name': 'A',
'P1 v P4': 0.3
}, {
'name': 'B',
'P1 v P4': 0.3
}, {
'name': 'C',
'P1 v P4': 0.3
}, {
'name': 'D',
'P1 v P4': 0.3
}, {
'name': 'E',
'P1 v P4': 0.3
}]
Now, I need to convert this to CSV such that there are column headers for each unique keys. In this case, I would have the following columns:
name, P1 v P2, P1 v P4, P2 v P3
So, I did the following:
from collections import ChainMap
items_dict = dict(ChainMap(*items))
headers = items_dict.keys()
with open(filename, 'w', encoding='utf8', newline='') as score_table:
csv_writer = csv.DictWriter(score_table, headers)
csv_writer.writeheader()
csv_writer.writerows(items)
But it's not giving me the result I want. I get something like this:
name, P1 v P2, P1 v P4, P2 v P3
A, 0.2
B, 0.3
C, 0.3
D, 0.3
E, 0.3
A, ,0.3
B, ,0.3
C, ,0.3
D, ,0.3
E, ,0.3
A, , ,0.9
B, , ,0.2
C, , ,0.3
D, , ,0.5
E, , ,0.9
What I wanted is that items be appended column-wise like the following:
name, P1 v P2, P1 v P4, P2 v P3
A, 0.2,0.3,0.9
B, 0.3,0.3,0.2
C, 0.3,0.3,0.3
D, 0.3,0.3,0.5
E, 0.3,0.3,0.9
But I have no idea how to get this. I've been trying several different things but without the desired results. I even tried converting this to pandas dataframe to see if it works, but that didn't do it either.
This should solve your problem:
df = pd.DataFrame(items).groupby("name").first()
It's grouping by name
and then getting the first non-null value for each column.
To save to a csv you should do:
df.to_csv('file_name.csv')