Accessing global variables from a python callback of C extension API

Question

I am new to python and C-extensions. I am writing a python code where I have created two threads and have defined a callback function py_cb(). In one thread I am appending to a global list after certain time intervals, whereas from the other thread, I am calling a C-extension library api. The C-extension api spawns a thread that calls the python callback function (py_cb) defined in my original file. In the callback function, I am trying to display the global list, but it seems it has a different id. I checked it using id(listName). Is there a way I can use the right global variable in the C-extension python callback function? I was assuming that global values will be shared across all the threads, is that not the case? Also, please suggest better solutions.

Following is the C code:

char *MODULE_NAME = "simple";
---

PyMODINIT_FUNC init_cmodule(void)
{
    PyObject *m = Py_InitModule3("_cmodule",module_methods,module_docstring);
    if (m == NULL)
    return;
}

static PyObject *cmodule_cmodule(PyObject *self, PyObject *args)
{
    int value = register_cb();
    PyObject *ret = Py_BuildValue("i",value);
    return ret;
}


void notifyFooHandle()
{
    printf("inside the notify_foo function\n");
    pModule = PyImport_ImportModule(MODULE_NAME);
    if (!pModule) {
        printf ("Failed to load the module\n");
        return;
    }
    PyObject *pFunc;
    pFunc = PyObject_GetAttrString(pModule, "py_cb");
    if (pFunc && PyCallable_Check(pFunc)) {
        PyObject_CallObject(pFunc,NULL);

    }
    else {
        Py_DECREF(pFunc);
        PyErr_Print();
        printf("Failed to send notification\n");
        return;
    }
    return;
}

void notify_foo(void)
{
    int t = 5;

    while (t < 10) {
        notifyFooHandle();
        sleep(5);
        t++;
    }
    return;
}

int register_cb(void)
{
    pthread_t notify_t;
    int rc = pthread_create(&notify_t, NULL, (void *) &notify_foo,NULL);
---
}

Following is the python callback API in simple.py file:

def py_cb():
    print "Received a call back from cmodule"
    global myList
    print "Global myList is ", id(myList)
    print myList
    return

Answer 1

Can you show us the code in your C library? If you're initializing a python VM there and then calling py_cb, it's easy to understand why the lists are different: you have two different python VM instances.

EDIT:

I think your problem is you're using two different python instances. First, you have your main program in python. In that instance you have a global "myList" so every function invoked from there will be accessing that particular instance of "myList". Then, you load a C module. When that C module opens you original python module in order to load py_cb, you are using a different python instance, you'll have a second "myList". In short, you have TWO different python instances running, the one you create when you run your main python script and one you create inside your C library to call py_cb.

If you want to share a common python instance, you'll have to create you instance in C, make it globally accessible in your C module, insert there you C functions and then run your main python function. When python calls to a C function, you'll be inside your original address space and not a new one, and when you make a call back to python you'll be using always the same python instance.