Search code examples
pythonexceptioncythoncythonize

Cython exception propagation for cdef functions returning multiple values


I have a cdef function returning an (int, int) tuple. I need to propagate exceptions and must therefore specify a return type for exceptions. Since my function never returns negative values, this could e.g. be (-1, -1). With the standard syntax specified in the documentation, my function would then look like

cdef (int, int) myFunction(int arg) except (-1, -1):
    ...

However, when cythonizing the above function, I get the error

Not allowed in a constant expression

I am aware that I could just turn on looking for exceptions after every function call via

cdef (int, int) myFunction(int arg) except *:
    ...

but this seems inefficient to me.

How can I propagate exceptions for functions with multiple return values?


Solution

  • The answer is: you cannot, at least not with the current version of Cython's cdef-tuples.

    The underlying issue, that for such tuples the operator == isn't defined by Cython (this could be done obviously by comparing every entry - but in the current version it is not defined).

    For a simple cdef int myfun() except * isn't actually that a big performance hit:

    • If result is not -1, there is almost no overhead (only comparison to -1) at all.
    • If result is -1, PyErr_Occurred is used to check for errors, which might mean quite overhead (all above for nogil-blocks)
    • one can choose a critical value X via except? X and thus minimize/optimize the necessary number of PyErr_Occured-calls.

    However, if Cython knows no ==-operator for a type (as with Cython's c-tuples), we are basically in cdef void myfun() except * case, that means there is no short-cut and PyErr_Occured must be always checked/called.


    You might say that == is defined for Cython's tuples, because it gets compiled. But if you look at the generated code, you will see, that for the comparison Cython's ctuples are converted to Python-tuples.


    I personally would first go with except * and see whether it really has a measurable impact. Because as cdef-function obviously can have Python-interaction, adding a little bit more might not hurt at all.

    If it does, the best is probably to change the signature of the function, for example

     cdef  int myFunction(int arg, (int, int) *a) except -1:
          ...
          return 0
    

    Even if this feels less smooth - it might be worth the hassle due to better performance.