C++-Python Interop: Marshalling data faster

Question

I'm working on an app that has a C++ and Python API, and am implementing an API call that needs to be used in both languages. Part of the data this API sends back is an array of structs representing color information:

struct {
    float r;
    float g;
    float b;
    float a;
} Foo_Pixel;

In my interop code, I marshal the data thusly:

Foo_Pixel color;

PyObject* tileValues = PyList_New( numValues );

for (int pixelIndex = 0; pixelIndex < numValues; pixelIndex++)
{
  color = value[pixelIndex];
  PyObject* newColourItem = Py_BuildValue("ffff", color.r, color.g, color.b, color.a);
  PyList_SetItem(tileValues, pixelIndex, newColourItem);
}

Essentially, I'm looping over the provided data and am building a list of tuples for the user to consume. However, this seems a little inefficient to me. Is there a faster way to create a PyObject out of an array of float structs?

Answer 1

Before Py_BuildValue can build the tuple you want, it needs to parse the format string you give it to determine the type and number of its arguments as well as what the returned PyObject will be. If you know all this ahead of time, you can skip these steps and build the tuple directly:

PyObject* newColourItem = PyTuple_New(4);
PyTuple_SET_ITEM(newColourItem, 0, PyFloat_FromDouble(color.r));
PyTuple_SET_ITEM(newColourItem, 1, PyFloat_FromDouble(color.g));
PyTuple_SET_ITEM(newColourItem, 2, PyFloat_FromDouble(color.b));
PyTuple_SET_ITEM(newColourItem, 3, PyFloat_FromDouble(color.a));

You'll notice too that I used PyTuple_SET_ITEM instead of PyTuple_SetItem. The two functions are the same except that the first does not perform any error checking. Thus, it is more efficient, but also dangerous if you give it a bad index. However, this is not a problem here since we know exactly what indexes we are filling.

On the same note, you could replace PyList_SetItem with PyList_SET_ITEM. Since you already know the indexes that you are using are valid, checking for errors is unnecessary here.

That is about all you can do however. There are no API functions for what you are doing and the code you posted was already pretty efficient to being with. If you are trying to improve the overall speed of your application, I would look elsewhere for optimizations.