Search code examples
pythonpandasgroup-byfillna

Groupby dataframe to get not null elements from each group member


I have a dataframe where in some cases a case has its records in more than one row, with nulls in some rows as so:

    date_rounded          1  2        3     4     5
0   2020-04-01 00:05:00 0.0  NaN     NaN    NaN   NaN   
1   2020-04-01 00:05:00 NaN  1.0     44.0   44.0  46.454    
2   2020-04-01 00:05:00 NaN  NaN     NaN    NaN   NaN   

I want to have only one row with the filled data, so far I have:

df.groupby(['date_rounded']).apply(lambda df0: df0.fillna(method='ffill').fillna(method='bfill').drop_duplicates())    

this works, but it is slow, any better ideas?

Thanks


Solution

  • You can also use groupby and first:

    df.groupby("date_rounded").first()
    
                           1    2     3     4       5
    date_rounded                                     
    2020-04-01 00:05:00  0.0  1.0  44.0  44.0  46.454