Search code examples
pandasdataframegroup-by

How to insert a row which is sum of groupby column into a dataframe (pandas)?


I want to group by name and sum then add at the beginning of each grouped data,like below: bold ones are sum values of grouped data..

df:

    name   place   eng_marks    math_marks
----------------------------------------------
0  prince  uk        70             80
1  prince  us        60             70
3  prince  fr        50             60
4  john    it        40             50
5  john    uk        30             40
6  lucy    np        20             10

Excepted df:

    name   place   eng_marks    math_marks
----------------------------------------------
0  prince  uk        180            210  ----> sum of groupby prince
1  prince  uk        70             80
2  prince  us        60             70
3  prince  fr        50             60
4  john    it        70             90   ----> sum of groupby john
5  john    it        40             50
6  john    uk        30             40
7  lucy    np        20             10   ----> sum of groupby lucy
8  lucy    np        20             10

I tried by grouping by name and sum the data(df) and stored in the new variable(df1) which then added into the original dataframe(df).i think this process is not efficient one.


Solution

  • group the dataframe by name and aggregate the marks columns with sum and place column with first then concat the aggregated dataframe with original dataframe and sort the index to maintain the order

    marks = (
        df.reset_index()
        .groupby('name', as_index=False, sort=False)
        .agg({'place': 'first', 'eng_marks': 'sum', 'math_marks': 'sum', 'index': 'first'})
    )
    
    # trick is to realign the index to maintain sort order
    marks.index = marks.pop('index')
    result = pd.concat([marks, df]).sort_index()
    

    Result

         name place  eng_marks  math_marks
    0  prince    uk        180         210
    1  prince    uk         70          80
    2  prince    us         60          70
    3  prince    fr         50          60
    4    john    it         70          90
    5    john    it         40          50
    6    john    uk         30          40
    7    lucy    np         20          10
    8    lucy    np         20          10