Search code examples
pythonmultithreadingqtqthread

Calling Py_Finalize returns assertation error


I am trying to call python code from c++ and I am using the following QThread class.

myclassName::myclassName() 
{
    Py_Initialize();
}
myclassName::~myclassName()
{
    Py_Finalize();
}

void myclassName::cpp_wrapper(string out1, string out2){

    ThreadState = PyEval_SaveThread();
    GILState = PyGILState_Ensure();

    Py_DECREF(PyImport_ImportModule("threading"));

    PyObject *moduleMain = PyImport_ImportModule("__main__");
    PyRun_SimpleString(
        "def wrapper(arg1, arg2) :          \n"\
        "   import sklearn                  \n"\
        "   print(arg1, arg2)               \n"\
        );
    PyObject *func = PyObject_GetAttrString(moduleMain, "wrapper");
    PyObject *args = PyTuple_Pack(2, PyUnicode_FromString(out1.c_str()), PyUnicode_FromString(out1.c_str()));

    Py_DECREF(moduleMain);
    Py_DECREF(func);
    Py_DECREF(args);

    PyGILState_Release(GILState);
    PyEval_RestoreThread(ThreadState);
}
void myclassName::run()
{
    algorithm_wrapper("Hello1", "Hello2");
    //here a sginal is emmited to the main thread function to delete myclassName* item.
}

int main(){
    myclassName* item = new myclassName();
    item->run();
}

The execution is fine (thanks to @Thomas in a previous post) but when Py_Finalize is called the following error is returned. The Py_Finalize is called when python code is executed and a signal is emitted to a slot in the main thread to delete the class object. I have also tried to initialize and finalize python (again by sending a signal) in the main thread but the same error returned.

Exception ignored in: <module 'threading' from 'C:\\Users\\username\\AppData\\Local\\Continuum\\Anaconda3\\Lib\\threading.py'>
Traceback (most recent call last):
  File "C:\Users\username\AppData\Local\Continuum\Anaconda3\Lib\threading.py", line 1289, in _shutdown
    assert tlock.locked()

Could ou please provide some help?

EDIT: After @Abid Rahman K's request.

.h file

class myclassName : public QThread
{
    Q_OBJECT
public:
    myclassName();
    ~myclassName();

protected:
    void run();

private:
    QMutex mutex_Python;

};

.cpp file

myclassName::myclassName():
mutex_Python(QMutex::Recursive)
{
    Py_Initialize();
}
myclassName::~myclassName()
{
    Py_Finalize();
}

void myclassName::cpp_wrapper(string out1, string out2){

    //ThreadState = PyEval_SaveThread();
    //GILState = PyGILState_Ensure();
    mutex_Python.lock();
    Py_DECREF(PyImport_ImportModule("threading"));

    PyObject *moduleMain = PyImport_ImportModule("__main__");
    PyRun_SimpleString(
        "def wrapper(arg1, arg2) :          \n"\
        "   import sklearn                  \n"\
        "   print(arg1, arg2)               \n"\
        );
    PyObject *func = PyObject_GetAttrString(moduleMain, "wrapper");
    PyObject *args = PyTuple_Pack(2, PyUnicode_FromString(out1.c_str()), PyUnicode_FromString(out1.c_str()));

    Py_DECREF(moduleMain);
    Py_DECREF(func);
    Py_DECREF(args);
    mutex_Python.unlock();
    //PyGILState_Release(GILState);
    //PyEval_RestoreThread(ThreadState);
}
void myclassName::run()
{
    algorithm_wrapper("Hello1", "Hello2");
    //here a sginal is emmited to the main thread function to delete myclassName* item.
}

int main(){
    myclassName* item = new myclassName();
    item->run();
}

Solution

  • As mentioned in Python official site about Py_Finalize:

    "Bugs and caveats: The destruction of modules and objects in modules is done in random order; this may cause destructors (del() methods) to fail when they depend on other objects (even functions) or modules. Dynamically loaded extension modules loaded by Python are not unloaded. Small amounts of memory allocated by the Python interpreter may not be freed (if you find a leak, please report it). Memory tied up in circular references between objects is not freed. Some memory allocated by extension modules may not be freed. Some extensions may not work properly if their initialization routine is called more than once; this can happen if an application calls Py_Initialize() and Py_Finalize() more than once."

    So you finalize ad dispose resources and processes manually before calling Py_Finalize