Search code examples
pythonc++address-sanitizer

ASAN memory leaks in embedded python interpreter in C++

Address Sanitizer is reporting a memory leak (multiple actually) originating from an embedded python interpreter when testing some python code exposed to c++ using pybind11.

I have distilled the code down to nothing other than calling PyInitialize_Ex and then PyFinalizeEx

#include <Python.h>

int main()
{
    Py_InitializeEx(0);
    Py_FinalizeEx();
    return 0;
}

All the memory leaks originate from a call to Py_InitializeEx.

Example:

Direct leak of 576 byte(s) in 1 object(s) allocated from:
    #0 0x7f0d55ce791f in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f0d55784a87  (/lib/x86_64-linux-gnu/libpython3.10.so.1.0+0x133a87)
    #2 0x7f0d55769e4f  (/lib/x86_64-linux-gnu/libpython3.10.so.1.0+0x118e4f)
    #3 0x7f0d5576a084  (/lib/x86_64-linux-gnu/libpython3.10.so.1.0+0x119084)
    #4 0x7f0d5576b0fa  (/lib/x86_64-linux-gnu/libpython3.10.so.1.0+0x11a0fa)
    #5 0x7f0d557974e6 in PyType_Ready (/lib/x86_64-linux-gnu/libpython3.10.so.1.0+0x1464e6)
    #6 0x7f0d5577fea6  (/lib/x86_64-linux-gnu/libpython3.10.so.1.0+0x12eea6)
    #7 0x7f0d5584eda5  (/lib/x86_64-linux-gnu/libpython3.10.so.1.0+0x1fdda5)
    #8 0x7f0d559697b3  (/lib/x86_64-linux-gnu/libpython3.10.so.1.0+0x3187b3)
    #9 0x7f0d558524e8 in Py_InitializeFromConfig (/lib/x86_64-linux-gnu/libpython3.10.so.1.0+0x2014e8)
    #10 0x7f0d558548fb in Py_InitializeEx (/lib/x86_64-linux-gnu/libpython3.10.so.1.0+0x2038fb)
    #11 0x55fe54d040ce in main /home/steve/src/test.cpp:8
    #12 0x7f0d54fe7d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #13 0x7f0d54fe7e3f in __libc_start_main_impl ../csu/libc-start.c:392
    #14 0x55fe54d04124 in _start (/home/steve/src/build/test+0x1124)

Questions:

  • How can I free up the memory so asan is happy?
  • Failing that, is it safe just to suppress Py_InitializeEx in its entirety?
Solution

  • Py_Initialize has a long lasting known issue, whose origin is in C source codes. For C++ programmers I recommend boost.python as a replacement that uses C++ features to simplify interoperability between the 2 languages.