Search code examples
pythonpython-2.7python-3.xmemory-profiling

A module to profile peak memory usage of Python code


Currently, I tried to use the memory_profiler module to get the used memory like the following code:

from memory_profiler import memory_usage
memories=[]
def get_memory(mem,ended):
  if ended:
    highest_mem=max(mem)
    print highest_mem
  else:
  memories.append(mem)

def f1():
  #do something
  ended=False
  get_memory(memory_usage(),ended)
  return #something
def f2():
  #do something
  ended=False
  get_memory(memory_usage(),ended)
  return #something

#main
f1()
f2()
ended=True
get_memory(memory_usage(),ended) #code end

>>>#output
# output 
# highest memory 

however, it did not successfully execute. It got stuck when ended=True and sent the value of memory_usage() and ended to the function of get_memory. It did not show any error as well., just waiting for long long time, then I force to stop executing. Anyone knows the better way or the solution?


Solution

  • An easy way to use memory_usage to get the peak / maximum memory from a block of code is to first put that code in a function, and then pass that function - without the () call - to memory_usage() as the proc argument:

    from memory_profiler import memory_usage
    
    def myfunc():
      # code
      return
        
    mem = max(memory_usage(proc=myfunc))
    
    print("Maximum memory used: {} MiB".format(mem))
    

    Other arguments allow you to collect timestamps, return values, pass arguments to myfunc, etc. The docstring seems to be the only complete source for documentation on this: https://github.com/fabianp/memory_profiler/blob/master/memory_profiler.py

    https://github.com/fabianp/memory_profiler/blob/4089e3ed4d5c4197925a2df8393d4cbfca745ae5/memory_profiler.py#L244