Efficiently execute mathematical Python expression in C++ many

Question

I have a python program, that generates a mathematical expression like

exp(sin(x-y))**2

Now I want to give this to my C++ program, that must evaluate this expression with different x,y values. My first approach was to use the Python.h library with PyRun_String.

Here the initialization code:

func=function;
Py_Initialize();
    memset(pythonString,0,strlen(pythonString));

    // add whiteNoise Kernel
    sprintf(pythonString,"from math import *;func=lambda x,y:(%s+0.1*(x==y))",function);
    //printf("%s\n",pythonString);
    PyRun_SimpleString(pythonString);

and here the code that gets evaluated many times:

char execString[200];
memset(execString,0,strlen(execString));

sprintf(execString,"result=func(%f,%f)",x1[0],  x2[0]);

PyObject* main = PyImport_AddModule("__main__");
PyObject* globalDictionary = PyModule_GetDict(main);
PyObject* localDictionary = PyDict_New();

//create the dictionaries as shown above

PyRun_String(execString, Py_file_input, globalDictionary, localDictionary);
double result = PyFloat_AsDouble(PyDict_GetItemString(localDictionary, "result"));

However, I think it's really too slow to parse the string with PyRun_String every time again. Is there a way to directly convert Python expressions to a C++ function, that can be invoked efficiently? Or is there any alternative? It would also be okay to use something like symbolicc++

Answer 1

I would suggest to pass all your inputs as arrays/vectors to your C++ code & get all solved at once. Also, try Py_CompileString & PyEval_EvalCode instead of PyRun_String. I had to solve millions of equations & found a 10X speed improvement.

Below is an example for a simple 'a + b' but with some more for loops one can generalize it for any equation with any number of variables. For a million values below is done in slightly less than a second on my machine (compared to 10 seconds for PyRun_String).

PyObject* main = PyImport_AddModule("__main__");
PyObject* globalDict = PyModule_GetDict(main);
PyCodeObject* code = (PyCodeObject*) Py_CompileString("a + b", "My Eqn", Py_eval_input);
for (/* millions of values in input */) {
    PyObject* localDict = PyDict_New();
    PyObject* oA = PyFloat_FromDouble(a);  // 'a' from input
    PyObject* oB = PyFloat_FromDouble(b);  // 'b' from input
    PyDict_SetItemString(localDict, "a", oA);
    PyDict_SetItemString(localDict, "b", oB);
    PyObject* pyRes = PyEval_EvalCode(code, globalDict, localDict);
    r = PyFloat_AsDouble(pyRes);
    // put r in output array
    
    Py_DECREF(pyRes);
    Py_DECREF(localDict)
}
Py_DECREF(code);