Sorting datetime objects by hour to a Pandas dataframe, then visualize to histogram with Matplotlib

Question

I need to sort viewers by hour to a histogram. I have some experience using Matplotlib to do that, but I can't find out what is the most pragmatic way to sort the dates by hour.

First I read the data from a JSON file, then store the two relevant datatypes in a pandas Dataframe, like this:

data = pd.read_json('data/data.json')

session_duration = pd.to_datetime(data.session_duration, unit='s').dt.time
time = pd.to_datetime(data.time, format='%H:%M:%S').dt.time

viewers = []

for x, y in zip(time, session_duration):
    viewers.append({str(x):str(y)})

EDIT: The source file looks like this, leaving out the irrelevant parts.

{
    "time": "00:00:09",
    "session_duration": 91
},
{
    "time": "00:00:16",
    "session_duration": 29
},
{
    "time": "00:00:33",
    "session_duration": 102
},
{
    "time": "00:00:35",
    "session_duration": 203
}

Note that the session_duration is in seconds.

I have to distinguish two types of viewers:

• Those who spent <= 1 minutes on the stream
• Those who spent >= 1 minutes on the stream

For that I do:

import datetime
for element in viewers:
    for time, session_duration in element.items():
        if datetime.strptime(session_duration, '%H:%M:%S').time() >= datetime.strptime('00:01:00', '%H:%M:%S').time():
            viewers_more_than_1min.append(element)
        else:
            viewers_less_than_1min.append(element)

As a result I have my values in a dictionary like this: {session_duration:time} Where the key is the time when the session ended the stream and the value is the time spent watching.

[{'00:00:09': '00:01:31'},
 {'00:00:16': '00:00:29'},
 {'00:00:33': '00:01:42'},
 {'00:00:35': '00:03:23'},
 {'00:00:36': '00:00:32'},
 {'00:00:37': '00:04:47'},
 {'00:00:47': '00:00:42'},
 {'00:00:53': '00:00:44'},
 {'00:00:56': '00:00:28'},
 {'00:00:58': '00:01:17'},
 {'00:01:04': '00:01:16'},
 {'00:01:09': '00:00:46'},
 {'00:01:29': '00:01:07'},
 {'00:01:31': '00:01:02'},
 {'00:01:32': '00:01:01'},
 {'00:01:32': '00:00:36'},
 {'00:01:37': '00:03:03'},
 {'00:01:49': '00:00:57'},
 {'00:02:01': '00:02:15'},
 {'00:02:18': '00:01:16'}]

As a final step I wish to create a histogram withMatplotlib representing the viewercount for each our from the two viewertypes mentioned above per hour. I assume it would go something like this:

import matplotlib.pyplot as plt
import datetime as dt
hours = [(dt.time(i).strftime('%H:00')) for i in range(24)]
plt.xlabel('Hour')
plt.ylabel('Viewer count')
plt.bar(hours, sorted_viewcount_byhour)

Answer 1

df = pd.read_json('data/data.json')

df['time'] = pd.to_datetime(df['time'])
#timedelta is a more appropriate data type for session_duration
df['session_duration'] = pd.to_timedelta(df['session_duration'], unit='s')

# Example filtering
df_short_duration = df[df['session_duration'].dt.total_seconds() <= 60]

# Example creating histogram
df_hist = df_short_duration.groupby(df['time'].dt.hour).count()
# Now just plot df_hist as a bar chart using matplotlib, might be something like plt.bar(df_hist.index, df_hist['count'])