Pandas groupby with identification of an element with max value in another column

Question

I have a dataframe with sales results of items with different pricing rules:

import pandas as pd
from datetime import timedelta
df_1 = pd.DataFrame()
df_2 = pd.DataFrame()
df_3 = pd.DataFrame()

# Create datetimes and data
df_1['item'] = [1, 1, 2, 2, 2]
df_1['date'] = pd.date_range('1/1/2018', periods=5, freq='D')
df_1['price_rule'] = ['a', 'b', 'a', 'b', 'b']
df_1['sales']= [2, 4, 1, 5, 7]
df_1['clicks']= [7, 8, 9, 10, 11]

df_2['item'] = [1, 1, 2, 2, 2]
df_2['date'] = pd.date_range('1/1/2018', periods=5, freq='D')
df_2['price_rule'] = ['b', 'b', 'a', 'a', 'a']
df_2['sales']= [2, 3, 4, 5, 6]
df_2['clicks']= [7, 8, 9, 10, 11]

df_3['item'] = [1, 1, 2, 2, 2]
df_3['date'] = pd.date_range('1/1/2018', periods=5, freq='D')
df_3['price_rule'] = ['b', 'a', 'b', 'a', 'b']
df_3['sales']= [6, 5, 4, 5, 6]
df_3['clicks']= [7, 8, 9, 10, 11]

df = pd.concat([df_1, df_2, df_3])
df = df.sort_values(['item', 'date'])
df.reset_index(drop=True)
df

It results with:

    item    date    price_rule  sales   clicks
0   1   2018-01-01       a       2       7
0   1   2018-01-01       b       2       7
0   1   2018-01-01       b       6       7
1   1   2018-01-02       b       4       8
1   1   2018-01-02       b       3       8
1   1   2018-01-02       a       5       8
2   2   2018-01-03       a       1       9
2   2   2018-01-03       a       4       9
2   2   2018-01-03       b       4       9
3   2   2018-01-04       b       5       10
3   2   2018-01-04       a       5       10
3   2   2018-01-04       a       5       10
4   2   2018-01-05       b       7       11
4   2   2018-01-05       a       6       11
4   2   2018-01-05       b       6       11

My goal is to:
1. group all items by day (to get a single row for each item and given day)
2. aggregate 'clicks' with "sum"
3. generate a "winning_pricing_rule" columns as following:
- for a given item and given date, take a pricing rule with the highest 'sales' value - in case of 'draw' (see eg: item 2 on 2018-01-03 in a sample above): choose just one of them (that's rare in my dataset, so it can be random...)

I imagine the result to look like this:

  item  date       winning_price_rule   clicks
0   1   2018-01-01      b               21
1   1   2018-01-02      a               24
2   2   2018-01-03      b               27  <<remark: could also be a (due to draw)
3   2   2018-01-04      a               30  <<remark: could also be b (due to draw)
4   2   2018-01-05      b               33

I tried:

a.groupby(['item', 'date'], as_index = False).agg({'sales':'sum','revenue':'max'})

but failed to identify a winning pricing rule.

Any ideas? Many Thanks for help :)

Andy

Answer 1

First convert column price_rule to index by DataFrame.set_index, so for winning_price_rule is possible use DataFrameGroupBy.idxmax - get index value by maximum sales in GroupBy.agg, because also is necessary aggregate sum:

df1 = (df.set_index('price_rule')
         .groupby(['item', 'date'])
         .agg({'sales':'idxmax', 'clicks':'sum'})
         .reset_index())

For pandas 0.25.+ is possible use:

df1 = (df.set_index('price_rule')
         .groupby(['item', 'date'])
         .agg(winning_pricing_rule=pd.NamedAgg(column='sales', aggfunc='idxmax'),clicks=pd.NamedAgg(column='clicks', aggfunc="sum'))
         .reset_index())