Call Python function with *args and **kwargs from C API

Question

I'm writing some C level function, it should accept a callable, *args and **kwargs, print something and call the callable with arguments. It's signature should be reflecting this one from Python:

def my_function(callable, *args, **kwargs): ...

Here's the C code that I have:

static PyObject *call_c(PyObject *self, PyObject *args, PyObject *keywords) {
    PyObject *f;
    PyObject *arguments;

    if (!PyArg_ParseTuple(args, "O|O", &f, &arguments))
    {
        PyErr_SetString(PyExc_ValueError, "invalid args");
        return NULL;
    }

    printf("Calling...\n");

    return PyObject_Call(f, arguments, keywords);
}

Here's how I'm trying to execute it:

def printer(value="test"):
    print(f"printed: {value}")

def call_c_wrapper(callable, *args, **kwargs):
    call_c(callable, args, **kwargs)

call_c(print, "some print value")
# Calling...
# Segmentation fault

call_c(printer, value="test")
# Traceback (most recent call last):
#   File "main.py", line 35, in <module>
#     call_c(printer, value="test")
# TypeError: call_c() takes no keyword arguments

call_c_wrapper(print, "some print value")
# > this is the only one that works:

# Calling...
# some print value

call_c_wrapper(printer, value="my value")
# > same as `call_c` with kwargs:

# Traceback (most recent call last):
#   File "main.py", line 37, in <module>
#     call_c_wrapper(printer, value="my value")
#   File "main.py", line 32, in call_c_wrapper
#     call_c(callable, args, **kwargs)
#  TypeError: call_c() takes no keyword arguments

What I'm doing wrong with the code? The *args parsing seems somewhat working, I'm OK with using a wrapper function if that's necessary, but why **kwargs doesn't work at all?

Edit

I've changed the PyMethodDef from METH_VARARGS to METH_VARARGS | METH_KEYWORDS, so call_c_wrapper() now works fine.
The call_c with args still gives me SegmentationFault, and the one with kwargs raises MemoryError.

Answer 1

As I suggested in the comments to your now-deleted earlier version of this question: I think PyArg_ParseTuple is the wrong tool for this because it doesn't really have a mechanism for handling *args.

What you want to do is something like the following Python code:

def f(*args, **kwds):
    f = args[0]
    args = args[1:]

    return f(*args, **kwds)

The easiest way of doing this is just to use the PyTuple_... API (https://docs.python.org/3/c-api/tuple.html).

Something along the lines of:

static PyObject *call_c(PyObject *self, PyObject *args, PyObject *keywords) {
    PyObject *f;
    PyObject *arguments;

    if (PyTuple_Size(args) == 0)
    {
        PyErr_SetString(PyExc_ValueError, "Missing argument 'f'");
        return NULL;
    }
    f = PyTuple_GET_ITEM(args, 0); // Note: borrowed reference so no reference counting needed
    arguments = PyTuple_GetSlice(args, 1, PyTuple_Size(args));
    // ideally you should check arguments isn't NULL too

    printf("Calling...\n");

    PyObject* result = PyObject_Call(f, arguments, keywords);
    Py_XDECREF(arguments);
    return result;
}