Search code examples
pythonpandasdictionarydata-munging

Transform complex column (dictionary-like) into multiple ones


I have a dataframe with two rows and single column:

import pandas as pd
raw_data = {'T': [[(0, 0.5), (1, 0.25), (2, 0.25)], [(1, 0.99)]]}
df = pd.DataFrame(raw_data, columns=['T'])

#T 
#[(0, 0.5), (1, 0.25), (2, 0.25)]
#[(1, 0.99)]

As you see row is kind of dictionary, I know the max number of dictionary elements (in our case 3 or 2 if you start counting from 0;) ). I want to create additional three columns T0, T1, T2 with following values within this dataframe:

T0   T1    T2 (df header)

0.5  0.25  0.25

0    0.99  0
  • there could be more than 2 rows
  • there could be more columns, but it shouldn't impact the problem

Solution

  • Use list comprehension with concat, then transpose, fillna and add_prefix:

    df=pd.concat([pd.Series(dict(x)) for x in raw_data['T']], axis=1).T.fillna(0).add_prefix('T')
    print (df)
        T0    T1    T2
    0  0.5  0.25  0.25
    1  0.0  0.99  0.00