Search code examples
pythoncythonnumba

Calling Cython functions from Numba jitted code


I know that a Numba-jitted function calling another jitted function will recognize this and automatically use a fast C calling convention rather than going through the Python object layer, and therefore avoid the high Python function call overhead:

import numba

@numba.jit
def foo(x):
    return x**2

@numba.jit
def bar(x):
    return 4 * foo(x)   # this will be a fast function call

My question is whether the same is true if I call a Cython function from Numba. So let's say I have a Cython module, foo.pyx:

cpdef double foo(double x):
    return x**2

As well as a standard Python module bar.py:

import numba
import foo

@numba.jit
def bar(x):
    return 4 * foo.foo(x)   # will this be a fast function call?

Will Numba recognize foo.foo as a C-callable function automatically or do I need to tell it manually by, say, setting up a CFFI wrapper?

EDIT: Upon further reflection, Cython functions are just standard "builtin" functions from the view of the Python interpreter. So the question can be made more general: does Numba optimize calls to builtin functions and methods to bypass the Python calling overhead?


Solution

  • There is a limited set of builtin functions (from both the python standard library and numpy) that numba knows how to translate into native code:

    Anything else will not be able to be jitted by Numba in nopython mode, thus resorting to objectmode which is much slower.

    There is no direct way to pass a cython function to Numba and have it be recognized in nopython mode. Numba does have hooks for cffi:

    http://numba.pydata.org/numba-doc/latest/reference/pysupported.html#cffi

    that can be leveraged to call outside C code, that you might be able to rig up to call cython if you could create a low level wrapper at the C-level; I'm not 100% sure if this is possible though. I wrote about doing this for calling RMath functions from Numba:

    https://web.archive.org/web/20160611082327/https://www.continuum.io/blog/developer-blog/calling-c-libraries-numba-using-cffi

    It might be helpful in getting you started if you go that route.