How to pass Numpy PyArray* via FFI

Idea is to be able to modify array from library, like an "output" from a function. Example:

ffi.cdef("""
    //Reads data from a file, and store in the numpy array
    void read_image(PyArray* arr);
""")

C = ffi.dlopen('libimage.so')
image = np.array([], dtype=np.float32)
C.read_image(image)
assert image.ndim == 2

Solution

You can't pass CPython-specific PyXxx structures via CFFI: you need to pass standard C data. Normally I'd answer that you need to design your cdef()'ed function with a standard C interface, for example something like:

ffi.cdef("""
    struct myimage_t {
        int width, height;
        float *data;
    };
    int read_image(struct myimage_t *output);  // fill in '*output'
    void free_image(struct myimage_t *img);   // free output->data
""")

myimage = ffi.new("struct myimage_t *")
if lib.read_image(myimage) < 0:
    raise IOError
...
lib.free_image(myimage)

Then you need to manually convert the myimage to a numpy array, somewhere in the "..." code above.

One better alternative is to use a Python callback: a callback that makes the numpy array according to spec and returns a C-standard float * pointer. The numpy array itself is saved somewhere in the callback. You could save it as a Python global, or more cleanly use a "handle" you pass via C. Requires the API version, not the ABI. In _example_build.py:

ffi.cdef("""
   extern "Python" float *alloc_2d_numpy_array(void *handle, int w, int h);
   void read_image(void *handle);
""")
ffi.set_source("_example_cffi", """
   void read_image(void *handle)
   {
       // the C code that eventually invokes
       float *p = alloc_2d_numpy_array(handle, w, h);
        // and then fill the data at 'p'
   }
""")
ffi.compile(verbose=True)

In file example.py:

from _example_cffi import ffi, lib

class Context:
    pass

@ffi.def_extern()
def alloc_2d_numpy_array(handle, w, h):
    context = ffi.from_handle(handle)
    context.image = np.ndarray([w, h], dtype=np.float32)
    return ffi.cast("float *", ffi.from_buffer(context.image))

context = Context()
lib.read_image(ffi.new_handle(context))
image = context.image