Search code examples
pythonpython-3.xintervalstiming

Measuring long time intervals in python


After searching about the different ways to measure time in Python 3.x, I've decided to use the perf_counter() function from the time module. When I tested it with a function that took some seconds to run it worked fine, but the code I wanna extract timing results from takes tens of hours to run.

Today I got the results and for my surprise the timing results were in the order of fractions of seconds, which is an absurd. Therefore, I come to ask you why does this happens and how to accurately measure long time intervals in Python 3.x.

The used code is as follows:

optimization_times = []
for i in range(30):
    # Search parameters on train set
    print("Round %d" % i)
    time_start = time.perf_counter()
    solution = metaheuristic.optimize()
    time_end = time.perf_counter()
    # Keep time spent
    optimization_times.append(time_end - time_start)

Operating system: Ubuntu 18.04

Thank you in advance.


Solution

  • You saw a wrap-around.

    From the docs:

    ... a clock with the highest available resolution to measure a short duration.

    If your interval is tens of hours, you should stick with time.time(), it has plenty of resolution for your needs.

    If you think you really need an insane number of digits of precision, then combine time.time() with perf_counter(), and also have a program on the side periodically record both values so you can sync the perf_counter offset to global time. Such a log corresponds to making your 48-bit counter a few bits wider, it lets you observe each wrap-around event.