Search code examples
pythonprofilermemory-profiling

log a report from memory_profiler

I am profiling my code using memory_profiler

from memory_profiler import profile

@profile
def whatever():
    ....
    ....

So, as many of you may know I am getting an output on the screen similar like this:

Line #    Mem usage  Increment   Line Contents
==============================================
     3                           @profile
     4      5.97 MB    0.00 MB   def my_func():
     5     13.61 MB    7.64 MB       a = [1] * (10 ** 6)
     6    166.20 MB  152.59 MB       b = [2] * (2 * 10 ** 7)
     7     13.61 MB -152.59 MB       del b
     8     13.61 MB    0.00 MB       return a

My question is:

Since the @profile process takes a lot of time, I was wondering if I could somehow log/store this output, and leave the script running, maybe during the night.

My idea is to use the decorator @profile in many def functions, and store all the results somehow in one single TXT or in many different TXT files, this is not importatnt, the important is if that would be possible.

Solution

  • As from the comments:

    If you just run

    run_my_thing > output.txt

    in the shell you can store stdout in a file.

    This will circumvent memory_profiler altogether. Obviously it's not ideal to just redirect stdout, but if it's for human analysis it shouldn't be a massive problem.