Search code examples
pythonnumpyyoctobitbake

Include numpy's arrayobject.h into bitbake recipe - how to fix installation order?


My python package depends on a c extension that, in turn, makes use of numpy's arrayobject.h. I built the image without the package and confirmed that this file exists: /usr/lib/python3.5/site-packages/numpy/core/include. I also patched distutil's setup.py as follows:

diff --git a/setup.py b/setup.py
index 99dcc2a..ecb2675 100644
--- a/setup.py
+++ b/setup.py
@@ -11,8 +11,12 @@ building_wheel = bool(sys.argv[1].strip() == 'bdist_wheel')


 def get_numpy_include():
-    import numpy
-    return numpy.get_include()
+    try:
+        import numpy
+        return numpy.get_include()
+
+    except ImportError:
+        return '/usr/lib/python3.5/site-packages/numpy/core/include'


 def get_build_include(lib_name):
@@ -106,6 +110,7 @@ setup(
             name='ringnes.ringbuffer_base',
             sources=sources,
             libraries=clibraries,
+            include_dirs=[get_numpy_include()],
             define_macros=[(sensor_type, 0)]),
         Extension(
             name='ringnes.mseed_ext',

Thus, the directory is hard coded but the fact that I have to catch the import exception indicates, that numpy is not available yet and hence the arrayobject.h is missing too.

So the question is: How do I ensure that numpy is there before bb works in this recipe?

This is (the important part) the recipe. Note the DEPENDS (thought that was enough):

inherit setuptools3

# Experimenting with CFLAGS
# TARGET_CFLAGS_append = " -I/usr/lib/python3.5/site-packages/numpy/core/include"

LAYERDEPENDS += " \ 
    meta-openembedded \
    meta-python \
"

DEPENDS += " \ 
    python3-numpy \
"

RDEPENDS_${PN} += " \
    python3-numpy \
    python3-scipy \
    python3-cryptography \
    python3-smbus \
    python3-psutil \
    python3-hbmqtt \
"

RRECOMMENDS_${PN} += " \
    python3-wifi \
"

Solution

  • The simple answer is to specify the dependency on the native (target) numpy:

    DEPENDS += " \ 
        python3-numpy-native \
    "
    

    I have not yet confirmed that everything builds till the end but at least arrayobject.h seems to be available now.

    EDIT: Everything seems to work fine now. Adding python3-numpy-native also made the patch for numpy obsolete.