Search code examples
datedatetimepandasseconds

Pandas - Convert HH:MM:SS.F string to seconds - Caveat : HH sometimes goes over 24H

I have the following dataframe :

**flashtalking_df =**

+--------------+--------------------------+------------------------+
| Placement ID | Average Interaction Time | Total Interaction Time |
+--------------+--------------------------+------------------------+
|      2041083 | 00:01:04.12182           | 24:29:27.500           |
|      2041083 | 00:00:54.75043           | 52:31:48.89108         |
+--------------+--------------------------+------------------------+

where 00:01:04.12182 = HH:MM:SS.F

I need to convert both columns, Average Interaction Time, and Total Interaction Time into seconds.

The problem is that Total Interaction Time goes over 24h.

I found the following code which works for the most part. However, when the Total Interaction Time goes over 24h, it gives me

ValueError: time data '24:29:27.500' does not match format '%H:%M:%S.%f'

This is the function I am currently using, which I grabbed from another Stack Overflow question, for both Average Interaction Time and Total Interaction Time:

flashtalking_df['time'] = flashtalking_df['Total Interaction Time'].apply(lambda x: datetime.datetime.strptime(x,'%H:%M:%S.%f'))
flashtalking_df['timedelta'] = flashtalking_df['time'] - datetime.datetime.strptime('00:00:00.00000','%H:%M:%S.%f')
flashtalking_df['Total Interaction Time'] = flashtalking_df['timedelta'].apply(lambda x: x / np.timedelta64(1, 's'))

If there's an easier way, please let me know.

Thank you for all your help

Solution

  • I think you need first convert to_timedelta and then to seconds by astype:

    df['Average Interaction Time'] = pd.to_timedelta(df['Average Interaction Time'])
                                   .astype('timedelta64[s]')
                                   .astype(int)

df['Total Interaction Time'] =   pd.to_timedelta(df['Total Interaction Time'])
                                   .astype('timedelta64[s]')
                                   .astype(int)
                                   .map('{:,.2f}'.format)
print (df)
   Placement ID  Average Interaction Time Total Interaction Time
0       2041083                        64              88,167.00
1       2041083                        54             189,108.00

    Solution with total_seconds, thank you NickilMaveli:

    df['Average Interaction Time'] = pd.to_timedelta(df['Average Interaction Time'])
                                   .dt.total_seconds()
                                   .map('{:,.2f}'.format)
df['Total Interaction Time'] =   pd.to_timedelta(df['Total Interaction Time'])
                                   .dt.total_seconds()
                                   .map('{:,.2f}'.format)

print (df)   
   Placement ID Average Interaction Time Total Interaction Time
0       2041083                    64.12              88,167.50
1       2041083                    54.75             189,108.89