How to return two PyObject lists from a C function in Python C API?

Question

I'm working with a C function that interfaces with Python using the Python C API. This function, PySolveProblem, currently returns a single PyObject list. However, I need to modify the function to return another list in addition to the first one.

Here is the current implementation of the function that returns the first list:

static PyObject *PySolveProblem(PyObject *self, PyObject *args)
{
    if(PyObject_Length(args) != 2) {
        PyErr_SetString(PyExc_TypeError, "Expected two arguments");
        return 0;
    }

    PyObject *arg1 = PyObject_GetItem(args, PyLong_FromLong(0));
    PyObject *arg2 = PyObject_GetItem(args, PyLong_FromLong(1));

    if(!PyUnicode_Check(arg1) || !PyUnicode_Check(arg2)) {
        PyErr_SetString(PyExc_TypeError, "Arguments must be strings");
        return 0;
    }

    const char *paramsStr = PyUnicode_AsUTF8(arg1);
    const char *problemStr = PyUnicode_AsUTF8(arg2);

    gbString params = gb_make_string(paramsStr);
    gbString problem = gb_make_string(problemStr);

    int tourSize = 0;
    int *tourPtr;
    int *finalHistory = 0;
    int finalHistorySize = 0;

    int jmpRes;
    jmpRes = setjmp(ErrorJumpBuffer);
    if(jmpRes != 0) {
        gb_free_string(params);
        gb_free_string(problem);
        return 0;
    }

    ElkaiSolveProblem(params, problem, &tourSize, &tourPtr, &finalHistorySize, &finalHistory);

    if(PyErr_Occurred() != 0) {
        return 0;
    }

    PyObject *list = PyList_New(tourSize);
    for (int i = 0; i < tourSize; i++) {
        PyObject *tourElement = PyLong_FromLong((long)(tourPtr[i]));
        PyList_SetItem(list, i, tourElement);
    }

    gb_free_string(params);
    gb_free_string(problem);

    return list;
}

I want to modify this function to also return historyList, a second list, and I tried to use a struct to hold both lists like this:

struct TupleLists {
    PyObject *a;
    PyObject *b;
};

Then, I tried to return this structure:

return (struct TupleLists) {list, historyList};

However, this results in the following errors:

warning C4047: 'initializing': 'Py_ssize_t' differs in levels of indirection from 'PyObject *'
warning C4133: 'initializing': incompatible types - from 'PyObject *' to 'PyTypeObject *'
error C2440: 'return': cannot convert from 'TupleLists' to 'TupleLists *'

I’m not sure how to correctly return multiple PyObject lists from this function. Should I be using a struct for this? If not, what is the recommended approach to return multiple lists from a C function to Python?

Answer 1

On a C language point of view, nothing is wrong in returning a struct... provided the function is declared to return that. This error:

C:\elkai-stats\elkai\elkai_elkai.c(84,5): error C2440: 'return': cannot convert from 'TupleLists' to 'TupleLists *' [C:\Users\Ђ«Ёб \AppData\Local\Temp\tmp9qhsfwne\build_elkai.vcxproj]

is an evidence that the function is declared to return a TupleLists *, probably with something close to:

static TupleLists *PySolveProblem(...

Just use static TupleLists PySolveProblem(... to get rid of that error. But the other warnings show that you have other problems.

'Py_ssize_t' differs in levels of indirection from 'PyObject *'

is a hint that you are trying to use a true Python object to initialize an integral variable which will use the address of the Python object instead of the integer value that is contains.

But IMHO you will end with a design problem. If the function is expected to be called from Python code, it shall return a PyObject *. So instead of returning a raw C struct, you could return a Python tuple object with:

return PyTuple_Pack(2, list, historyList);