Search code examples
pythonsetuptoolspyopencl

pip install pyopencl fails with ImportError: No module named 'numpy'


When installing pyopencl with pip, it complains about missing numpy:

Collecting pyopencl
Downloading https://files.pythonhosted.org/packages/bc/58/3ab1246e94986f1b6953e76d7ea7e69d2dbfef7b3f3874eded48524a024f/pyopencl-2018.2.2.tar.gz (341kB)
    100% |████████████████████████████████| 348kB 9.5MB/s 
    Complete output from command python setup.py egg_info:
    ---------------------------------------------------------------------------
    Pybind11 is not installed.
    ---------------------------------------------------------------------------
    Very likely, the build process after this message will fail.

    Simply press Ctrl+C and type
    python -m pip install pybind11
    to fix this. If you don't, the build will continue
    in a few seconds.

    [1] https://pybind11.readthedocs.io/en/stable/
    ---------------------------------------------------------------------------
    Continuing in 1 seconds...    
    ---------------------------------------------------------------------------
    Mako is not installed.
    ---------------------------------------------------------------------------
    That is not a problem, as most of PyOpenCL will be just fine
    without it. Some higher-level parts of pyopencl (such as
    pyopencl.reduction) will not function without the templating engine
    Mako [1] being installed. If you would like this functionality to
    work, you might want to install Mako after you finish
    installing PyOpenCL.

    Simply type
    python -m pip install mako
    either now or after the installation completes to fix this.

    [1] http://www.makotemplates.org/
    ---------------------------------------------------------------------------
    Hit Ctrl-C now if you'd like to think about the situation.
    ---------------------------------------------------------------------------
    Continuing in 1 seconds...   
    Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-nccshu4t/pyopencl/setup.py", line 353, in <module>
        main()
    File "/tmp/pip-install-nccshu4t/pyopencl/setup.py", line 320, in main
        language='c++',
    File "/tmp/pip-install-nccshu4t/pyopencl/aksetup_helper.py", line 41, in __init__
        self._include_dirs = self.include_dirs
    File "/tmp/pip-install-nccshu4t/pyopencl/aksetup_helper.py", line 55, in get_include_dirs
        return self._include_dirs + self.get_additional_include_dirs()
    File "/tmp/pip-install-nccshu4t/pyopencl/aksetup_helper.py", line 52, in get_additional_include_dirs
        return [self.get_numpy_incpath()]
    File "/tmp/pip-install-nccshu4t/pyopencl/aksetup_helper.py", line 47, in get_numpy_incpath
        file, pathname, descr = find_module("numpy")
    File "/home/app/nb-ocl/.venv/lib/python3.5/imp.py", line 296, in find_module
        raise ImportError(_ERR_MSG.format(name), name=name)
    ImportError: No module named 'numpy'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-nccshu4t/pyopencl/

This is an issue with the latest version of pyopencl published in PyPI. The latest version in the github repo already addresses this issue, using the new ExtensionUsingNumpy instead of the old NumpyExtension in setup.py::setup().

Still, if Mako and pybind11 dependencies are not installed, the installation of pyopencl wil fail. Why aren't these handled by setup.py/setuptools as well?


Solution

  • Since there are no binaries (wheels) available in PyPI, pip install defaults to pulling the software distribution (sdist) archive and tries to build it. The building process of pyopencl depends on numpy, pybind11 and mako, and for that reason pip fails.

    OTOH if pyopencl provided binaries (wheels) this issue could be circumvented. There is a pull request awaiting review that should help with that: https://github.com/inducer/pyopencl/pull/264