Search code examples
pythonmemoryallocation

python memory allocation trace


My code allocates continously memory (~12kb per second). With a runtime of 8 hours its a lot of memory!!

Because of this I want to trace the moments/code lines my python code allocates memory.

Something like you can do with processed code lines with:

python -m trace --count -C ./tmp code.py

this generate a view where you can see how often this line was executed. It looks like:

code.cover

1:     import sys
1:     import os
1534:  while 1:
1534:      print "foo"

I need this for memory allocation. if possible something like

1245 B    import sys
893 B     import os
17.46 KB  import somecode

Solution

  • Looks like this question has already been answered here: Python memory profiler

    Maybe this one can help you: http://pypi.python.org/pypi/memory_profiler

    From the docs, execute the code passing the option -m memory_profiler to the python interpreter to load the memory_profiler module and print to stdout the line-by-line analysis. If the file name was example.py, this would result in:

    $ python -m memory_profiler example.py
    

    Output will follow:

    Line #    Mem usage  Increment   Line Contents
    ==============================================
         3                           @profile
         4      5.97 MB    0.00 MB   def my_func():
         5     13.61 MB    7.64 MB       a = [1] * (10 ** 6)
         6    166.20 MB  152.59 MB       b = [2] * (2 * 10 ** 7)
         7     13.61 MB -152.59 MB       del b
         8     13.61 MB    0.00 MB       return a