Search code examples
clinkercythondistutils

How to link custom C (which itself needs special linking options to compile) with Cython?


so I'm trying to create a Cython module which uses a .c file that I've written. That .c file needs a special linking option (to compile it, I need gcc -o mycode mycode.c -lfftw3f). I could potentially just rewrite my .c file in Cython, but I want to know how to do this.

I am using fftw3, and when compiling, you need to use the -lfftw3f option IN ADDITION to #include <fftw3.h> in the .c file if you want to use the float version, which I do.

My setup.py looks like the following:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

sourcefiles = ['mycode_caller.pyx', 'mycode.c']

ext_modules = [Extension("myext", sourcefiles, libraries=['fttw3f'])]

setup(
  name = 'My Extension',
  cmdclass = {'build_ext': build_ext},
  ext_modules = ext_modules
)

I made a header file called mycode.h which looks like the following, and includes a prototype for the transform() function, which is defined in mycode.c:

#include <fftw3.h>
#include <math.h>

#ifndef FOURIER_H_INCLUDED
#define FOURIER_H_INCLUDED
fftwf_complex** transform(float** in, int length);
#endif

And my Cython file, mycode_caller.pyx looks like this:

import numpy as np
cimport numpy as np

cdef extern from "stdlib.h":
    void free(void* ptr)
    void* malloc(size_t size)

cdef extern from "fftw3.h":
    struct fftwf_complex:
        pass

cdef extern from "fourier.h":
    fftwf_complex** transform(float** in_arr, int length)

cdef float** npy2c_float2d(np.ndarray[float, ndim=2] a):
    cdef float** a_c = <float**>malloc(a.shape[0] * sizeof(float*))
    for k in range(a.shape[0]):
        a_c[k] = &a[k, 0]
    return a_c

cpdef test_transform(data):
    nparr = np.zeros([14, 31])
    c_array = npy2c_float2d(nparr)
    ans = transform(c_array, 31)

When I run python setup.py build_ext --inplace, it builds fine, but if I try to import it, it will claim the following:

ImportError: ./myext.so: undefined symbol: fftwf_execute

This error happens as a result of not having the -lfftw3f option passed to gcc during compilation. How do I resolve this? There's no way to specify linker commands in a .c source file right? Do I need to tell Cython.Distutils to use this option somehow? Thank you for your help!

EDIT: So, I added libraries=[fttw3f] to my setup.py file, and now it throws an error on build:

gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro build/temp.linux-x86_64-2.7/emodsp.o build/temp.linux-x86_64-2.7/fourier.o -lfftw3f -o /home/carson/Documents/Caltech/Senior/Winter/art89/Project/openepoc/emodsp.so
/usr/bin/ld: /usr/local/lib/libfftw3f.a(alloc.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/libfftw3f.a: could not read symbols: Bad value
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

Solution

  • Just use the libraries option to Extension:

    Extension("myext", sourcefiles, libraries = ['fftw3f'], library_dirs = ['/path/to/fftw/libs'])