Search code examples
pythonpandasdataframenested

How to split a nested list inside a Pandas Dataframe column vertically,ie, into separate rows


I have a Dataframe, I want to split Co2 rowise :

Co1 Co2
1 [{'link':'http:abc.com'},{'link':'http:def.com'}]
2 [{'link':'http:ghi.com'},{'link':'http:lmn.com'}]

Expected output

Co1 Co2
1 http:abc.com
1 http:def.com
2 http:ghi.com
2 http:lmn.com

Solution

  • Use concat with dicionary of DataFrame created in list comprehension and join by DataFrame.join to original, DataFrame.pop is used for extract column:

    import ast
    
    #if values are string repr of lists
    dfs = {k:pd.DataFrame(ast.literal_eval(x)) for k, x in df.pop('Co2').items()}
    #if values are lists of dicts
    #dfs = {k:pd.DataFrame(x) for k, x in df.pop('Co2').items()}
    
    df = df.join(pd.concat(dfs).reset_index(level=1, drop=True)).reset_index(drop=True)
    print (df)
       Co1          link
    0    1  http:abc.com
    1    1  http:def.com
    2    2  http:ghi.com
    3    2  http:lmn.com