Search code examples
pythonmemoryplotprofilermemory-profiling

Python memory_profiler inconsistent plots

I have recently started using the python memory profiler from here. As a test run, I tried to profile the toy code from here following the instructions therein. I have some naive questions on the outputs I saw.

import time

@profile
def test1():
    n = 10000
    a = [1] * n
    time.sleep(1)
    return a

@profile
def test2():
    n = 100000
    b = [1] * n
    time.sleep(1)
    return b

if __name__ == "__main__":
    test1()
    test2()

This is the output using mprof run and then plot command line options: With function profiling

After removing the @profile lines, I ran the profiler again and obtained the following result: Without function profiling

Except for the brackets for the functions, I was expecting almost identical plots (since the code is simple), but I am seeing some significant differences such as the ending time of the plot, variations within brackets etc.

Can someone please shed light into these differences?

Edit: For small intervals, the plot with function profiling looks like: small intervals

Solution

  • The differences you are seeing are probably due to the fact that the information stored by @profile is counted within the total memory used by the program. There is also a slight overhead of storing this information, hence the different running times.

    Also, you might get slightly different plots in different runs just due to variations in how Python manages memory.