How to extend a Python list with an array of PyObjects

Question

I'm trying to deal with an API that returns an array of PyObjects and I want to extend an existing python list with it's contents.

So far I'm just appending every item of the array:

static PyObject *
myfunc(PyObject *module, PyObject *alist, PyObject **items, Py_ssize_t arrsize)
{
    Py_ssize_t i;
    for (i = 0 ; i < arrsize; i++) {
        if (PyList_Append(alist, items[i]) == -1) {
            return NULL;
        }
    }
    return alist;
}

But a list is more or less just a wrapper around an array of PyObjects itself.

So is there a better (or faster) way to extend that list? I haven't found a function in the Python-C-API and given the overallocation of PyList_Object it seems hacky to create a new array, then memcpy the original arrays and then just assign it to ((PyList_Object *)alist)->ob_item.

Answer 1

Your intention is implementing or using an list.extend(), but

l.extend(items)

is just equivalent to

l[len(l):] = items

This can be done within the python c api PyList_SetSlice itself, though you'll have the overhead of creating a new list. But it's better than resizing the list with every append.

static PyObject *
myfunc(PyObject *module, PyObject *alist, PyObject **items, Py_ssize_t arrsize)
{
    Py_ssize_t i, len = PyList_GET_SIZE(alist);
    PyObject *l = PyList_New(arrsize);
    int r = PyList_SetSlice(alist, len, len + arrsize, l);
    Py_CLEAR(l);
    if (r == -1) {
        return NULL;
    }

    for (i = 0 ; i < arrsize; i++) {
        Py_INCREF(items[i]);
        PyList_SET_ITEM(alist, len + i, items[i]);
    }
    return Py_INCREF(alist), alist;
}

Additionally you were loosing a reference on return.