Search code examples
python-2.7pipcentos7hdf5netcdf4

hdf5 dependencies trying to install netcdf4


I am trying to install NetCDF4 via pip on a CentOS 7 system running Python 2.7. The following HDF5 packages have already been installed via yum:

[user@host]$ rpm -qa | grep hdf5
hdf5-openmpi-1.8.12-12.el7.x86_64
hdf5-1.8.12-11.el7.x86_64

Despite the packages being installed, I still encounter this error trying to install hdf5:

[user@host]$ pip install -r requirements.txt 
.
.
.
Collecting netCDF4>=1.1.1 (from -r requirements.txt (line 9))
  Using cached https://files.pythonhosted.org/packages/79/0d/caa957cc1b42b718ce4b9b3e849e6f7aa99faad2d522d8f2d7a33500fba0/netCDF4-1.5.6.tar.gz
    ERROR: Command errored out with exit status 1:
     command: /home/ooiui/miniconda2/envs/ooiui279/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-cOs19I/netCDF4/setup.py'"'"'; __file__='"'"'/tmp/pip-install-cOs19I/netCDF4/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
         cwd: /tmp/pip-install-cOs19I/netCDF4/
    Complete output (30 lines):
    Package hdf5 was not found in the pkg-config search path.
    Perhaps you should add the directory containing `hdf5.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'hdf5' found
    reading from setup.cfg...
    
        HDF5_DIR environment variable not set, checking some standard locations ..
    checking /home/ooiui/miniconda2/envs/ooiui279/include ...
    hdf5 headers not found in /home/ooiui/miniconda2/envs/ooiui279/include
    checking /home/ooiui/miniconda2/envs/ooiui279/Library/include ...
    hdf5 headers not found in /home/ooiui/miniconda2/envs/ooiui279/Library/include
    checking /home/ooiui/include ...
    hdf5 headers not found in /home/ooiui/include
    checking /usr/local/include ...
    hdf5 headers not found in /usr/local/include
    checking /sw/include ...
    hdf5 headers not found in /sw/include
    checking /opt/include ...
    hdf5 headers not found in /opt/include
    checking /opt/local/include ...
    hdf5 headers not found in /opt/local/include
    checking /usr/include ...
    hdf5 headers not found in /usr/include
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-cOs19I/netCDF4/setup.py", line 425, in <module>
        _populate_hdf5_info(dirstosearch, inc_dirs, libs, lib_dirs)
      File "/tmp/pip-install-cOs19I/netCDF4/setup.py", line 366, in _populate_hdf5_info
        raise ValueError('did not find HDF5 headers')
    ValueError: did not find HDF5 headers
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

In the past, when I installed NetCDF4 on CentOS 6, I didn't have to make any special tweaks. Has something changed that requires additional prep? Thanks in advance.

Thanks to phd, I had the HDF5 headers installed and set the HDF5_DIR environment variable to /include to reflect the install location. Now I seem to be running into a similar roadblock with NetCDF4 itself:

Collecting netCDF4>=1.1.1 (from -r requirements.txt (line 9))
  Using cached https://files.pythonhosted.org/packages/79/0d/caa957cc1b42b718ce4b9b3e849e6f7aa99faad2d522d8f2d7a33500fba0/netCDF4-1.5.6.tar.gz
    ERROR: Command errored out with exit status 1:
     command: /home/ooiui/miniconda2/envs/ooiui279/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-l853Xs/netCDF4/setup.py'"'"'; __file__='"'"'/tmp/pip-install-l853Xs/netCDF4/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
         cwd: /tmp/pip-install-l853Xs/netCDF4/
    Complete output (23 lines):
    Package hdf5 was not found in the pkg-config search path.
    Perhaps you should add the directory containing `hdf5.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'hdf5' found
    reading from setup.cfg...
    checking /usr/include ...
    HDF5 library version: 1.8.12 headers found in /usr/include
    HDF5 library version: 1.8.12 found in /usr
    
    NETCDF4_DIR environment variable not set, checking standard locations..
    checking /home/ooiui/miniconda2/envs/ooiui279 ...
    checking /home/ooiui/miniconda2/envs/ooiui279/Library ...
    checking /home/ooiui ...
    checking /usr/local ...
    checking /sw ...
    checking /opt ...
    checking /opt/local ...
    checking /usr ...
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-l853Xs/netCDF4/setup.py", line 441, in <module>
        raise ValueError('did not find netCDF version 4 headers')
    ValueError: did not find netCDF version 4 headers
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

We are trying to install NetCDF4 via pip per the requirements.txt file, but it appears to be failing on missing NetCDF4 headers.


Solution

  • ValueError: did not find HDF5 headers

    The error means there is no development files. You have the library but not headers. Install:

    yum install hdf5-devel