Search code examples
pythonc++memory-leakspython-c-apicpython

Memory leak when embedding python into my application


The following program, when linked against python 2.7.13 and run on Windows 10 slowly but steadily leaks memory.

#include <Python.h>
#include <iostream>

int main()
{
    std::cout << "Python version: " << PY_VERSION << std::endl;

    while (true)
    {
        Py_Initialize();
        //PyGC_Collect();
        Py_Finalize();
    }

    return 0;
}

The interesting fact is that it seems not every iteration leaks memory. What I see, though, is that the reference count that python prints slowly increases by a (non-constant) count of approximately 90 per iteration regardless of the leak. Using the Visual Studio Diagnostic Tools I figured out that the leak is coming from a call to PyImport_ImportModule() when it reads a compiled module from disk (the actual call stack is several levels deep).

Are any additional cleanup steps necessary that I am not aware of? Or is there something about the Python garbage collector that might cause this and it is not a "real" memory leak?


Solution

  • Py_Finalize — Python/C API Reference Manual (emphasis mine):

    <...>
    Bugs and caveats: The destruction of modules and objects in modules is done in random order; this may cause destructors (__del__() methods) to fail when they depend on other objects (even functions) or modules. Dynamically loaded extension modules loaded by Python are not unloaded. Small amounts of memory allocated by the Python interpreter may not be freed (if you find a leak, please report it). Memory tied up in circular references between objects is not freed. Some memory allocated by extension modules may not be freed. Some extensions may not work properly if their initialization routine is called more than once; this can happen if an application calls Py_Initialize() and Py_Finalize() more than once.