I'm trying to build a python extension which uses the numpy C-API to manipulate numpy arrays. While setting up a deployment chain, I encountered a problem.
In my requirements.txt and setup.py I have added the dependency numpy>=1.7, because I'm using API features which have been introduced in that version. I'm building wheels in the quay.io/pypa/manylinux1_x86_64 docker image. Inside the image, I'm installing my requirements using pip. This installs numpy==1.14, because this is the current version, which matches my dependencies.
However, when I install the mypackage-xxx-manylinux_x84_64.whl on my ubuntu machine (which has numpy 1.8), I receive the following error upon importing my package
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
RuntimeError: module compiled against API version 0xc but this version of numpy is 0x9
---------------------------------------------------------------------------
ImportError
[...]
ImportError: numpy.core.multiarray failed to import
The obvious fix is that I run pip install -U numpy. However, I don't want to tell every user of my package to manually run this command if their numpy library is <1.14 (even if it matches my dependency requirements). There are several issues about this with exactly this recommendation (e.g. 1, 2). I'm asking here from the perspective of a package developer. What can I do to prevent this in the first place?
What is the best practice here? Should I specifically add a dependency for numpy >= 1.14, such that this version is automatically installed? However, when numpy releases a new version, my package is automatically built against the new one in the docker image, which leads to the same problem. Or should I explicitly install numpy==1.14 in the docker image, to have a fixed version and add this as a dependency?
Or is there a way to annotate wheels, against which python packages they have been built?
P.S. Of course, there is no problem with source installation, because then the package is built against the numpy version which is installed on the user's system.
I've solved this issue to my satisfaction by adding a minimal 'pyproject.toml' with an exact pinned numpy version. This makes pip install in the the PEP 517 compliant build isolation mode where ONLY the dependencies listed in this file are installed.
[build-system]
requires = ['numpy==1.12.2', 'setuptools>=40.8.0', 'wheel', 'packaging']
build-backend = "setuptools.build_meta"
My install_requires in my setup.py still lists numpy>=1.12.2 though, so when the produced wheel gets installed, pip will not (try to) forcefully downgrade numpy to 1.12.2 if for example 1.16.2 is already installed.