I have the following dataframe:
import pandas as pd
hits = {'id': ['A','A','A','A','A','A','B','B','B','C','C','C'],
'datetime': ['2010-01-02 03:00:00','2010-01-02 03:00:14','2010-01-02 03:00:35','2010-01-02 03:00:38',
'2010-01-02 03:29:10','2010-01-02 03:29:35','2010-01-02 03:45:20','2010-01-02 06:10:05',
'2010-01-02 06:10:15','2010-01-02 07:40:15','2010-01-02 07:40:20','2010-01-02 07:40:25'],
'uri_len': [10,20,25,15,20,10,20,25,15,30,40,45]
}
df = pd.DataFrame(hits, columns = ['id', 'datetime','uri_len'])
df['datetime'] = pd.to_datetime(df['datetime'], format='%Y-%m-%d %H:%M:%S')
print (df)
id datetime uri_len
0 A 2010-01-02 03:00:00 10
1 A 2010-01-02 03:00:14 20
2 A 2010-01-02 03:00:35 25
3 A 2010-01-02 03:00:38 15
4 A 2010-01-02 03:29:10 20
5 A 2010-01-02 03:29:35 10
6 B 2010-01-02 03:45:20 20
7 B 2010-01-02 06:10:05 25
8 B 2010-01-02 06:10:15 15
9 C 2010-01-02 07:40:15 30
10 C 2010-01-02 07:40:20 40
11 C 2010-01-02 07:40:25 45
I want to group the hits by sessions, using id as the grouping by variable. For me, a session is an inactivity period of more than 15 seconds (calculated from the datetime column), or a decrease of the uri_len column, and in both cases comparing consecutive hits.
I know how to group by each condition individually:
df['session1'] = (df.groupby('id')['datetime']
.transform(lambda x: x.diff().gt('15Sec').cumsum())
)
df['session2'] = (df.groupby('id')['uri_len']
.transform(lambda x: x.diff().lt(0).cumsum())
)
Is there a way to combine both transformations in the same line, so the output is directly this?:
id datetime uri_len session
0 A 2010-01-02 03:00:00 10 0
1 A 2010-01-02 03:00:14 20 0
2 A 2010-01-02 03:00:35 25 1
3 A 2010-01-02 03:00:38 15 2
4 A 2010-01-02 03:29:10 20 3
5 A 2010-01-02 03:29:35 10 4
6 B 2010-01-02 03:45:20 20 0
7 B 2010-01-02 06:10:05 25 1
8 B 2010-01-02 06:10:15 15 2
9 C 2010-01-02 07:40:15 30 0
10 C 2010-01-02 07:40:20 40 0
11 C 2010-01-02 07:40:25 45 0
If I understood correctly, you want to add them?
df['session'] = df.groupby('id')['datetime'].transform(lambda x:
x.diff().gt('15Sec').cumsum()) + df.groupby('id')['uri_len'].transform(lambda x:
x.diff().lt(0).cumsum())
a more clear way:
s1 = df.groupby('id')['datetime'].transform(lambda x:
x.diff().gt('15Sec').cumsum())
s2 = df.groupby('id')['uri_len'].transform(lambda x: x.diff().lt(0).cumsum())
df['session'] = s1+s2