I have a Python package containing a number of C/C++ extensions built as a single wheel. I'm trying to understand how to ensure the wheel and shared libraries it contains correctly advertise that they use the stable ABI at a particular API version. I build the package using a setup.py that I run this way.
% python setup.py bdist_wheel --py-limited-api=cp34
I think the cp34 part is how I indicate that I'm using the stable ABI and at most the Python 3.4 API. The resulting wheel is named goober-1.2-cp34-abi3-linux_x86_64.whl. The highlighted part shows the Python and ABI tags. Without the --py-limited-api, that part is cp38-cp38, matching my Python 3.8. Is that enough to advertise that my wheel should work with all Python 3.x starting from 3.4, without recompiling? I guess I'd specify cp3 to indicate all 3.x versions.
For the shared libraries, I compile the C/C++ source this way.
% gcc ... -DPy_LIMITED_API=0x03040000 ... blooper.c
In this case, the shared library is named blooper.cpython-38-x86_64-linux-gnu.so, with nothing indicating it supports the stable ABI and the 3.4 API. From PEP 3149 I expected to see that somewhere in the name. Otherwise, won't Python 3.8 be the only version willing to import this module?
Thanks.
It might be surprising, but adding --py-limited-api=cp34 only changes the name of the wheel, but not its content - i.e. it will be still "usual" version pinned to the Python version which with it has been built.
The first step is to create a setup.py which would produce a C-extension which uses stable-API and which declares it as well. To my knowledge distutils has no support for stable C-API, so setuptools should be used.
There is a minimal example:
from setuptools import setup, Extension
my_extension = Extension(
name='foo',
sources = ["foo.c"],
py_limited_api = True,
define_macros=[('Py_LIMITED_API', '0x03040000')],
)
kwargs = {
'name':'foo',
'version':'0.1.0',
'ext_modules': [my_extension],
}
setup(**kwargs)
Important details are:
py_limited_api should be set to True, thus the resulting extension will have the correct suffixes (e.g. abi3), once build.Py_LIMITED_API macro should be set to the correct value, otherwise non-stable or wrong stable C-API will be used.The resulting suffix of the extension might also be surprising. The CPython documentation states:
On some platforms, Python will look for and load shared library files named with the abi3 tag (e.g. mymodule.abi3.so). It does not check if such extensions conform to a Stable ABI. The user (or their packaging tools) need to ensure that, for example, extensions built with the 3.10+ Limited API are not installed for lower versions of Python.
"Some platforms" are Linux and MacOS, one can check it by looking at
from importlib.machinery import EXTENSION_SUFFIXES
print(EXTENSION_SUFFIXES)
# ['.cpython-38m-x86_64-linux-gnu.so', '.abi3.so', '.so'] on Linux
# ['.cp38-win_amd64.pyd', '.pyd'] on Windows
that means on Linux, the result will be foo.abi3.so and just foo.pyx on Windows (see e.g. this code in setuptools).
Now, just running
python setup.py bdist_wheel
would build an extension, which could be used with any Python version>= 3.4, but pip would not install it for anything else than CPython-3.8 with pymalloc on Linux (because the name of wheel is foo-0.1.0-cp38-cp38m-linux_x86_64.whl). This is the part, from the documentation, where the packaging system needs to ensure, that it doesn't come to version mismatch.
To allow pip to install for multiple python versions, the wheel should be created with --py-limited-api-version:
python setup.py bdist_wheel --py-limited-api=cp34
due to the resulting name foo-0.1.0-cp34-abi3-linux_x86_64.whl, pip will know, it is safe to install for CPython>=3.4.
To make clear: CPython doesn't really know, that the c-extension with suffix abi3.so (or .pyx on Windows) can really be used by the interpreter (it just assumes in good faith) - it is pip who ensures, that right version is installed.