Search code examples
pythonctypes

Python ctypes: pass argument by reference error


I have a C++ function that I want you call in Python 2.7.12, looking like this:

extern "C" {
    double* myfunction(double* &y, double* &z, int &n_y, int &n_z, int a, int b)
    {
        vector<double> _x;
        vector<double> _y;
        vector<double> _z;

        // Call some external C++ function
        cpp_function(_x, _y, _z, a, b);

        // Convert vectors back to arrays
        double* x = &_x[0]; // or x = _x.data();
        y = &_y[0];
        z = &_z[0];
        n_y = static_cast<int>(_y.size());
        n_z = static_cast<int>(_z.size());
        return x;
    }
}

Basically this function takes as input two integers a,b (plus some other data that I omitted for clarity purpose) and do some calculations before putting the results into two arrays y, z and their respective sizes into n_y, n_z, and returning an array x of size a*b.

After building this function to a shared library myfunction.so, I call it in Python as follows:

from ctypes import *

libc = CDLL('myfunction.so')
myfunction = libc.myfunction

myfunction.restype = POINTER(c_double)
myfunction.argtypes = [POINTER(c_double), POINTER(c_double),
                       c_int, c_int,
                       c_int, c_int]

y = POINTER(c_double)()
z = POINTER(c_double)()
n_y = c_int()
n_z = c_int()

a = 18
b = 18
x = myfunction(byref(y), byref(z),
               byref(n_y), byref(n_z),
               c_int(a), c_int(b))

Running this script I obtained an error:

ctypes.ArgumentError: argument 3: : wrong type

So the c_int type of n_y is not correct. What should I put instead?

Thank you very much for your help!


UPDATE

Following the suggestion by @GiacomoAlzetta and @CristiFati, I have changed my code to use pointers instead of pass by reference, as follows.

(y and z are similar so let me omit z)

extern "C" {
    double* myfunction(double** y, int* n_y, int a, int b)
    {
        vector<double> _x;
        vector<double> _y;

        // Call some external C++ function
        cpp_function(_x, _y, a, b);

        // Convert vectors back to arrays
        double* x = &_x[0]; // or x = _x.data();
        *y = &_y[0];
        *n_y = static_cast<int>(_y.size());
        return x;
    }
}

Now in C++, I call the above function as follows:

double* y;
int n_y;
int a = 18;
int b = 18;
double* x = myfunction(&y, &n_y, a, b);

which works. And in Python:

from ctypes import *

libc = CDLL('myfunction.so')
myfunction = libc.myfunction

myfunction.restype = POINTER(c_double)
myfunction.argtypes = [POINTER(POINTER(c_double)), POINTER(c_int),
                       c_int, c_int]

y = POINTER(POINTER(c_double))()
n_y = POINTER(c_int)()

a = 18
b = 18
x = myfunction(y, n_y, c_int(a), c_int(b))

which produced a Segmentation fault error, which happened at the line

*y = &_y[0];

Thank you for your help!


Solution

  • You're almost there.
    In the meantime, stay close to [Python.Docs]: ctypes - A foreign function library for Python.

    Remember that you should handle pointer arguments (actually it applies to all of them, but for non pointer ones things are straightforward) the same way, no matter where you are.

    In other words, what you do in C (instantiate a variable and pass its pointer to the function), you should also do in Python (instead of instantiate the variable pointer and pass it to the function).

    Translated into code, you should modify the way you initialize y, n_y, and the function (myfunction) call:

    >>> #from ctypes import *  # Anti-pattern. Don't use it
    >>> import ctypes as cts
    >>>
    >>> y = cts.POINTER(cts.c_double)()  # Instantiate simple pointer
    >>> n_y = cts.c_int()
    >>> a = 18
    >>> b = 18
    >>> x = myfunction(cts.pointer(y), cts.pointer(n_y), a, b)
    

    Notes:

    • What I stated in a comment (Undefined Behavior because vectors are living on the stack and will be destroyed when exiting the function) still stands. To fix it either:
      • Allocate the data on heap (malloc / new) before returning it (when done with it, you'll also need to deallocate it (free / delete), to avoid memory leaks)
      • Make them static

    Some (remotely connected) examples: