Verifying Python reference count when sending list as parameter

Question

I have a C++ class that imports a Python module as a file and continuously calls a function for it before returning the received data:

ProgramLiveProcess::ProgramLiveProcess() {
    //Start Python to run the Program Python code
    Py_Initialize();

    //Import the Program live processing module
    program_live_processing_module = PyImport_ImportModule(MODULE_NAME);

    //Get the objects for the functions to call
    live_process = PyObject_GetAttrString(program_live_processing_module, "Program_LiveProcess");
}

//Creates Python list from intensities array
PyObject* ProgramLiveProcess::get_intensities_list(unsigned int intensities[INTENSITIES_DATA_SIZE]) {
    PyObject* tr = PyList_New(0);
    for (int i = 0; i < INTENSITIES_DATA_SIZE; i++) {
        PyObject* ta = PyLong_FromLong(intensities[i]);
        PyList_Append(tr, ta);
    }
    return tr;
}

//TODO: Make this actually work
//Frees objects in intensities Python list and the list itself
void ProgramLiveProcess::free_intensities_list(PyObject* i_list) {
    //std::cout << std::to_string(i_list->ob_refcnt) << std::endl;
    for (int i = 0; i < INTENSITIES_DATA_SIZE; i++) {
        PyObject* ta = PyList_GET_ITEM(i_list, i);
        Py_DECREF(ta);
    }
    Py_DECREF(i_list);
}

processed_data* ProgramLiveProcess::send_to_program_live_process(unsigned int intensities[INTENSITIES_DATA_SIZE], bool use_sensor_1) {
    //Call the Program_LiveProcess function
    PyObject* intensities_py = get_intensities_list(intensities);
    PyObject* calculate_args = PyTuple_Pack(2, intensities_py, use_sensor_1 ? Py_True : Py_False); //True or false depending on if using sensor 1 or 2
    PyObject* processed_data_tuple = PyObject_CallObject(live_process, calculate_args);
    Py_DECREF(calculate_args);
    free_intensities_list(intensities_py);

    //Get the data from the function
    PyObject* s0p = PyTuple_GetItem(processed_data_tuple, 0);
    PyObject* s0t = PyTuple_GetItem(processed_data_tuple, 1);

    //Return a struct containing the data
    processed_data* to_return = (processed_data*)malloc(PROCESSED_DATA_STRUCT_SIZE);
    if (to_return == NULL)
        return to_return;
    to_return->sensor_1_pressure = (float)PyFloat_AS_DOUBLE(s0p);
    to_return->sensor_1_temperature = (float)PyFloat_AS_DOUBLE(s0t);

    //Free python objects and return
    Py_DECREF(s0p);
    Py_DECREF(s0t);
    return to_return;
}

//Finalize the interpreter after freeing objects
ProgramLiveProcess::~ProgramLiveProcess() {
    Py_DECREF(live_process);
    free(live_process);
    Py_Finalize();
}

While the program receives data fine, I am getting inconsistencies in regards to memory leaks and crashes when run under different conditions. While I can't give a stack trace, I am wondering if I am doing something wrong in regards to creating or dereferencing Python objects. Often, the program only runs when I comment out the free_intensities_list call.

Thanks in advance.

Answer 1

In your free_intensities_list function, you may be doing double frees. Consider using the Py_XDECREF instead.

void Py_DECREF(PyObject *o)
Decrement the reference count for object o. The object must not be NULL; if you aren’t sure that it isn’t NULL, use Py_XDECREF(). If the reference count reaches zero, the object’s type’s deallocation function (which must not be NULL) is invoked.

Source: https://docs.python.org/3/c-api/refcounting.html

Also, you can use MALLOC_CHECK to check for double frees.