Replace a function pointer in a shared library with ctypes

Question

I am trying to replace an existing function pointer in a shared library with a callback defined in Python, through ctypes.

The source of the shared library in C:

#include <assert.h>
#include <stdio.h>

void (*plot)();

int c_main(int argc, void** argv) {
  printf("plot is %p\n", (void*)plot);
  assert(plot != NULL);
  plot();
  return 0;
}

The source of the Python script:

from sys import platform
from pathlib import Path
import ctypes
import _ctypes


FUNCTYPE = ctypes.WINFUNCTYPE if platform == 'win32' else ctypes.CFUNCTYPE


def dlclose(obj):
    if platform == "win32":
        _ctypes.FreeLibrary(obj._handle)
    else:
        _ctypes.dlclose(obj._handle)


def enc_args(args):
    C_ARGS = (ctypes.POINTER(ctypes.c_char) * len(args))()
    for idx, arg in enumerate(args):
        C_ARGS[idx] = ctypes.create_string_buffer(arg.encode("utf-8"))
    return C_ARGS


@FUNCTYPE(None)
def plotxy():
    print("plotxy")


C_ARGS = enc_args([str(Path(__file__))])
CAUXDLL = ctypes.CDLL("./test.so")

print(plotxy)

print(CAUXDLL.plot)

CAUXDLL.plot = plotxy

print(CAUXDLL.plot)

print(CAUXDLL.c_main(len(C_ARGS), C_ARGS))

The script to test it:

gcc -fPIC -shared -o test.so test.c
python3 test.py

The output I get:

# ./test.sh
<CFunctionType object at 0x7fb1f0abb1c0>
<_FuncPtr object at 0x7fb1f0abb640>
<CFunctionType object at 0x7fb1f0abb1c0>
plot is (nil)
python3: test.c:8: c_main: Assertion `plot != NULL' failed.
./test.sh: line 3: 21171 Aborted                 python3 test.py

So, it seems that the function defined in Python (plotxy) is of type CFunctionType, while the function pointer defined in C is of type _FuncPtr. Although the replacement in CAUXDLL is applied, it seems to have no effect when the main function is called.

Apart from reading https://docs.python.org/3/library/ctypes.html#module-ctypes, I found other questions (e.g. How to use typedef in ctypes or python cytpes creating callback function - Segmentation fault (core dumped)), but I cannot find how to convert CFunctionType (plotxy) to _FuncPtr.

EDIT

I believe this might not be an issue with the regular usage of ctypes. That's something I have successfully achieved and which is sufficiently explained in the docs. This question goes beyond. I don't want to execute the C function. I want Python to replace an existing function pointer with a callback written in Python. Note that it is possible to do it by using a helper C function (see https://github.com/ghdl/ghdl-cosim/blob/master/vhpidirect/shared/pycb/caux.c#L32-L40). Hence this question is about how to achieve it without that helper function (if possible).

Answer 1

The way to access global variables from ctypes is to use in_dll, but there doesn't seem to be an exposed way to change a function pointer. I was only able to read it and call it, so I don't think it is possible without a helper function.

The example below alters an int global variable, but CFUNCTYPE instances don't have a value member to alter it. I added a C helper to set the global to work around the issue and a default value of the callback to verify it was accessed correctly before changing it.

test.c:

#include <stdio.h>

#define API __declspec(dllexport)

typedef void (*CB)();

void dllplot() {
    printf("default\n");
}

API CB plot = dllplot;
API int x = 5;

API int c_main() {
  printf("x=%d (from C)\n",x);
  plot();
  return 0;
}

API void set_cb(CB cb) {
    plot = cb;
}

test.py:

from ctypes import *

PLOT = CFUNCTYPE(None)

dll = CDLL('./test')
dll.c_main.argtypes = ()
dll.c_main.restype = c_int
dll.set_cb.argtypes = PLOT,
dll.set_cb.restype = None

@PLOT
def plotxy():
    print("plotxy")

x = c_int.in_dll(dll,'x')
plot = PLOT.in_dll(dll,'plot')
print(f'x={x.value} (from Python)')
x.value = 7
print('calling plot from Python directly:')
plot()
print('calling c_main():')
dll.c_main()
dll.set_cb(plotxy)
print('calling plot from Python after setting callback:')
plot()
print('calling plot from C after setting callback:')
dll.c_main()

Output:

x=5 (from Python)
calling plot from Python directly:
default
calling c_main():
x=7 (from C)
default
calling plot from Python after setting callback:
plotxy
calling plot from C after setting callback:
x=7 (from C)
plotxy

Note that global pointers use .contents to access their value, so I experimented with using a POINTER(CFUNCTYPE(None)) and using plot.contents = plotxy but that doesn't assign the global variable correctly and C crashed.

I even tried adding an actual global pointer to function pointer:

API CB plot = dllplot;
API CB* pplot = &plot;

and then using:

PLOT = CFUNCTYPE(None)
PPLOT = POINTER(PLOT)
plot = PPLOT.in_dll(dll,'pplot')
plot.contents = plotxy

That let me assign the function via .contents, but c_main still called the default plot value. So the functionality of using CFUNCTYPE as anything but a function parameter doesn't appear to be implemented.