PyObject_CallObject crashed when called out of main function scope

Question

I'm building a simple module to wrap a C function. The main function of this module (test_wrapper) basically receives a python function and call it:

#include <Python.h>

static PyObject* test_wrapper(PyObject* self, PyObject* args) {
    PyObject* py_handler;
    int args_ok = PyArg_ParseTuple(args, "O", &py_handler);

    PyObject_CallObject(py_handler, NULL);

    return Py_BuildValue("i", 0);
}

static PyMethodDef TestModuleMethods[] = {
    { "test", test_wrapper, METH_VARARGS, NULL },
    { NULL, NULL, 0, NULL }
};

static struct PyModuleDef TestModule = {
    PyModuleDef_HEAD_INIT,
    "test_module",
    NULL,
    -1,
    TestModuleMethods
};

PyMODINIT_FUNC PyInit_test_module(void) {
    return PyModule_Create(&TestModule);
}

The code above works fine. The thing is, let's suppose I need to call the passed python function (py_handler) in the future in another way, by a signal handler, for example, and now it expects an integer as an argument:

PyObject* py_handler;

void handler(int signo) {
    PyObject* handler_args = PyTuple_Pack(1, PyLong_FromLong(signo));
    PyObject_CallObject(py_handler, handler_args); //seg fault
}

static PyObject* test_wrapper(PyObject* self, PyObject* args) {
    int args_ok = PyArg_ParseTuple(args, "O", &py_handler);
    //Py_INCREF(py_handler); //adding this didn't work

    //calls sigaction to set handler function

    return Py_BuildValue("i", 0);
}

By doing this, PyObject_CallObject crashes (seg fault) when it's called by handler.

What could I be missing here?

If relevant, I'm building the .so with setup.py.

Answer 1

Acquiring and releasing GIL was enough to solve the problem:

void handler(int signo) {
    PyGILState_STATE state = PyGILState_Ensure();
    PyObject* handler_args = PyTuple_Pack(1, PyLong_FromLong(signo));
    PyObject_CallObject(py_handler, handler_args);
    PyGILState_Release(state);
}

Wrapping code in PyGILState_STATE object and after execution releasing it ensures that only one thread at a time is executing the Python code. In general, it's a good practice to acquire the GIL before calling any Python API function and release it afterwards.