Call python script from c++ without loading the file each time

Question

I have a python script that I need to call lots of times (around 160000 times) and while I can do this in less that a second with hard C++ code, if I try to load and call a python script to do this, it will probably take hours! I think that if I loaded the script once, then run the loaded script over and over, it will be significantly faster. Unfortunately, I don't know how to do this. I don't think I can load the file with ifstream, then use PyRun_SimpleString on all the lines of the string. In case it isn't faster however, is it possible to return a 2D array in python, then convert that Array into a std::vector?

Answer 1

Consider the following function, in a file called multiply.py

#!/usr/bin/python3

def do_multiply(a, b):
    c = 0
    for i in range(0, a):
        c += b
    return c

In your C++ file:

// Load the function 
PyObject *pName     = PyUnicode_FromString("multiply");
PyObject *pModule   = PyImport_Import(pName);
PyObject *pFunction = PyObject_GetAttrString(pModule, "do_multiply")

Let's say you want to call this function 3 times, as follows:

struct NumberPair 
{ 
    int x;
    int y;
};

std::vector<NumberPair> numberPairs { {1, 2}, {5, 6}, {10, 12} };

Then, you can simply call PyObject_CallFunction several times, while the module is loaded:

for(const auto &numberPair : numberPairs)
{
    PyObject *product = PyObject_CallFunction(pFunction, "ii", numberPair.x, numberPair.y);
    if(product != NULL)
    {
        std::cout << "Product is " << PyLong_AsLong(product) << '\n';
        Py_DECREF(product);
    }
}

There is no need to close the module between calls to PyObject_CallFunction, so that shouldn't be a problem.