python multithreading ctypes variadic-functions

How to call a variadic function with ctypes from multiple threads?

I have a shared library, libfoo.so, with a variadic function:

int foo(int handle, ...);

that uses handle to access to static variables within the library.

Now, I want to use it with ctypes in a multithread program.

import ctypes as ct

# main
lib = ct.cdll.LoadLibrary('libfoo.so')
foo = lib.foo
foo.restype = ct.c_int

# thread 1 code
def thread1(handle):
    foo.argtypes = [ct.c_int, ct.c_int]
    foo(handle, 2);

# thread 2 code
def thread2(handle):
    foo.argtypes = [ct.c_int, ct.c_double]
    foo(handle, 2.);

The problem is that both threads modify the same foo.argtypes and this leads to conflicts. I cannot load the same library twice because I need to access to static data into the library. Moreover, the foo object, that is an instance of _FuncPtr, is not copyable.

An obvious solution is to add a mutex to protect argtypes while foo is being called. Are there any other solutions to this problem?

Solution

Instead of setting .argtypes for each function resulting in a race condition, create the correct ctypes type as you call each function:

import ctypes as ct

# main
lib = ct.cdll.LoadLibrary('libfoo.so')
foo = lib.foo
foo.argtypes = ct.c_int,  # define the known types. ctypes will allow more
foo.restype = ct.c_int

# thread 1 code
def thread1(handle):
    foo(handle, ct.c_int(2))

# thread 2 code
def thread2(handle):
    foo(handle, ct.c_float(2))

Here's a test. The C printf aren't serialized so may mix up two prints in a single line, but the numbers are correct:

test.c

#include <stdio.h>
#include <stdarg.h>

#ifdef _WIN32
#   define API __declspec(dllexport)
#else
#   define API
#endif

API int foo(int handle, ...) {
    va_list valist;
    va_start(valist, handle);
    switch(handle) {
    case 1:
        printf("%d\n", va_arg(valist, int));
        break;
    case 2:
        printf("%f\n", va_arg(valist, float));
        break;
    case 3:
        int x = va_arg(valist, int);
        float y = va_arg(valist, float);
        printf("%d %f\n", x, y);
        break;
    default:
        ;
    }
    va_end(valist);
    return 123;
}

test.py

import ctypes as ct
from threading import Thread

dll = ct.CDLL('./test')
dll.foo.argtypes = ct.c_int,
dll.foo.restype = ct.c_int

def thread1():
    for _ in range(5):
        dll.foo(1, ct.c_int(1));

# thread 2 code
def thread2():
    for _ in range(5):
        dll.foo(2, ct.c_float(2.125))

def thread3():
    for _ in range(5):
        dll.foo(3, ct.c_int(3), ct.c_float(3.375))

threads = [Thread(target=f) for f in (thread1, thread2, thread3)]
for t in threads:
    t.start()
for t in threads:
    t.join()

Output: