Search code examples
pythonvirtualenv

What is the prefered way of generating extension module filename suffix in virtualenvs


We have a Python project that has a C++ extension module based on pybind11. We use python3-config --extension-suffix to get the filename suffix of the extension module. It works fine on systems with only one primary version of Python installed.

Now we are trying to build the extension module for multiple Python versions within a single system. So we created a system with multiple Python versions installed, each with a corresponding virtual environment. Then the problem emerges: python3-config is not available in virtual environments so the version of executed python3-config might not match the version of activated virtualenv. For example, python3-config --extension-suffix might output .cpython-311-x86_64-linux-gnu.so even if a python3.8 virtualenv is activated.

The only way I can think of is to copy the corresponding version's python3-config to the virtualenv's bin directory but then we have to remember to do this every time a new virtualenv is created otherwise the wheel generated is not going to work. So I wonder if there is a better way and what people usually do when they build extension modules inside virtualenvs.


Solution

  • I have encountered the same problem and here is my solution.

    You can use Python's built-in sysconfig module to get the filename suffix of the extension module. To that end, you need to call sysconfig.get_config_var("EXT_SUFFIX") from python.

    Therefore running python -c 'import sysconfig; print(sysconfig.get_config_var("EXT_SUFFIX"))' will return the wanted filename suffix, because it will use the Python version of the virtual environment:

    ❯ python -c 'import sysconfig; print(sysconfig.get_config_var("EXT_SUFFIX"))'
    .cpython-312-x86_64-linux-gnu.so