Compile file with two separate libraries in Cython

Question

I wrote a library in Cython that has two different "modes":

1. If rendering, I compile using GLFW.
2. If not rendering, I compile using EGL, which is faster, but I have not figured out how to render with it.

What is the recommended way to handle this situation?

Right now, I have the following directory structure:

mujoco
├── __init__.py
├── simEgl.pyx
├── simGlfw.pyx
├── sim.pxd
└── sim.pyx

simEgl.pyx contains EGL code and simGlfw.pyx contains GLFW code. setup.py uses an environment variable to choose one or the other for the build.

This works ok, except that I need to recompile the code every time I want to switch between modes. There must be a better way.

Update

I agree that the best approach is to simultaneously compile two different libraries and use a toggle to choose which one to import. I already do have a base class in sim.pyx with shared functionality. However this base class must itself be compiled with the separate libraries. Specifically, sim.pyx depends on libmujoco.so which depends on either GLFW or EGL.

Here is my exhaustive search of possible approaches:

1. If I do not compile an extension for sim.pyx, I get ImportError: No module named 'mujoco.sim'
2. If I compile an extension for sim.pyx without including graphics libraries in the extension, I get ImportError: /home/ethanbro/.mujoco/mjpro150/bin/libmujoco150.so: undefined symbol: __glewBlitFramebuffer
3. If I compile an extension for sim.pyx and choose one set of graphics libraries (GLFW), then when I try to use the other set of graphics libraries (EGL) this does not work either unsurprisingly: ERROR: GLEW initalization error: Missing GL version
4. If I compile two different versions of the sim.pyx library, one with one set of libraries, one with the other, I get: TypeError: unorderable types: dict() < dict() which is not a very helpful error message, but appears to result from trying to share a source file between two different extensions.

Something like option 4 should be possible. In fact, if I were working in raw C, I would simply build two shared objects side by side using the different libraries. Any advice on how to get around this Cython limitation would be very welcome.

Answer 1

(This answer is just a summary of the comments with a bit more explanation.)

My initial suggestion was to create two extension modules defining a common interface. That way you pick which to import in Python but be able to use them in the same way once imported:

if rendering:
   import simGlfw as s
else:
   import simEgl as s
s.do_something() # doesn't matter which you imported

It appears from the comments that the two modules also share a large chunk of their code and its really just the library that they're linked with that defines how they behave. Trying to re-use the same sources with

Extension(name='sim1', sources=["sim.pyx",...)
Extension(name='sim2', sources=["sim.pyx",...)

fails. This is because Cython assumes that the module name will be the same as the filename, and so creates a function PyInit_sim (on Python 3 - Python 2 is named slightly differently but the idea is the same). However, when you import sim1.so it looks for the function PyInit_sim1, fails to find it, and gives an error.

An easy way round it is to put the common code in "sim.pxi" and use Cython's largely obsolete include mechanism to textually include that code in sim1.pyx and sim2.pyx

include "sim.pxi"

Although include is generally no longer recommended and cimport is preferred since it provides more "Python-like" behaviour, include is a simple solution to this particular problem.