Search code examples
pythonperformancescipyf2py

Limitations of scipy.weave for more complex performance code


  1. Is there any overhead of calling functions using weave.inline() in comparison to direct call of native (C, Fortran) static compiled function? I was worrying that weave.inline() does compile the code every time the function is called (which it does not, fortunately ), but still I worry that it may slow down by doing some tests if the code is already compiled or not (I don't know how it works inside ).
  2. Is it possible to write also more complex (multifunction) C code using weave.inline()? I tried that and I got error a function-definition is not allowed here before '{' token
  3. Is there any possibility how to compile the code in such a way, that I can call the functions independently.

Bacground / application / example

I have a N-Body probem solver using f2py which consist of several functions ( one function evaluate forces between bodies, other functions integrate differential equations by RKF45 ).

Because I find it hard to make f2py work under Windows and I want make multiplatform python library, I'm trying to rewrite it to C and use scipy.weave based on this tutorial http://wiki.scipy.org/PerformancePython

For illustration, I made example with much simpler differential equation dA(t)/dt = -k*A(t)

from scipy import weave
from scipy.weave import converters
from pylab import *
n = 100
As = zeros(n);
def testODE(A, k, n):
    code = r"for (int i=0; i<n; i++) { As(i)=A; A -= (double)k*(double)A; }"
    return weave.inline(code, ['A','k','n','As'], type_converters=converters.blitz, compiler = 'gcc')               
ks = arange(0.02,0.2,0.02)              
for k in ks:
    testODE( 10.0, k, n)
    plot ( As );
show();

This works just fine. I'm just not sure if there is some overhead due to weave.inline() calls.

However, If I make a separate function for evaluation of derivatives ( like the force in N-Body problem ) than I get error a function-definition is not allowed here before '{' token

def testODE_multiFunc(A, k, n):
    code = r'''
        double evalDerivatives(double a, double b){ return a*b; }
        for (int i=0; i<n; i++) { As(i)=A; A -= evalDerivatives(k,A); }
        '''
    return weave.inline(code, ['A','k','n','As'], type_converters=converters.blitz, compiler = 'gcc', verbose=3)    

Solution

  • Here's an explanation of how compiled code is cached: the weave catalog

    If you want to inline complex (multi-function) C code, you could define your functions in a separate header file (say, foo.h) and then pass that to the call to inline:

    weave.inline(code, ['A','k','n','As'], headers=['"foo.h"'],
                 type_converters=converters.blitz)
    

    Note the strange quoting. From the docs:

    headers : [str], optional A list of strings specifying header files to use when compiling the code. The list might look like ["","'my_header'"]. Note that the header strings need to be in a form than can be pasted at the end of a #include statement in the C++ code.