Search code examples
pythonubuntuhdf5caffepycaffe

Unable to find hdf5 libraries when building pycaffe


When running

make pycaffe

We are ending up with missing hdf5 libraries.

CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp
/usr/bin/ld: cannot find -lhdf5_hl
/usr/bin/ld: cannot find -lhdf5
collect2: error: ld returned 1 exit status
Makefile:502: recipe for target 'python/caffe/_caffe.so' failed
make: *** [python/caffe/_caffe.so] Error 1

Note that every conceivable combination of these libraries has been installed on this ubuntu 15.10 system.

$ dpkg --get-selections | grep hdf5
hdf5-helpers                    install
libhdf5-10:amd64                install
libhdf5-cpp-10:amd64                install
libhdf5-dev                 install
libhdf5-mpi-dev                 install
libhdf5-mpich-10:amd64              install
libhdf5-mpich-dev               install
libhdf5-openmpi-10:amd64            install
libhdf5-openmpi-dev             install
libhdf5-serial-dev              install

Now I realize that is overkill .. but they were added one at a time - and then we retried the make pycaffe before adding another.. We just wanted to be sure missing hdf5 libs were not the core issue.

Update In response to the supplied answer from @jon

 dpkg -L libhdf5-dev | grep so
/usr/lib/x86_64-linux-gnu/libhdf5_cpp.so
/usr/lib/x86_64-linux-gnu/libhdf5_serial.so
/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5_cpp.so
/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5_hl.so
/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so
/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5_fortran.so
/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5_hl_cpp.so
/usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5hl_fortran.so
/usr/lib/x86_64-linux-gnu/libhdf5_hl_cpp.so
/usr/lib/x86_64-linux-gnu/libhdf5_serial_fortran.so
/usr/lib/x86_64-linux-gnu/libhdf5_serial_hl.so
/usr/lib/x86_64-linux-gnu/libhdf5_serialhl_fortran.so

And in the Makefile.config:

LIBRARY_DIRS := /usr/opt/libsr $(PYTHON_LIB) 
   /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu

Solution

  • run dpkg -L libhdf5-dev | grep so should output:

    /usr/lib/x86_64-linux-gnu/libhdf5_hl.so
    /usr/lib/x86_64-linux-gnu/libhdf5hl_fortran.so
    /usr/lib/x86_64-linux-gnu/libhdf5_fortran.so
    /usr/lib/x86_64-linux-gnu/libhdf5_hl_cpp.so
    /usr/lib/x86_64-linux-gnu/libhdf5_cpp.so
    /usr/lib/x86_64-linux-gnu/libhdf5.so
    

    Make sure the path /usr/lib/x86_64-linux-gnu (which contains libhdf5.so and libhdf5_hl.so) is in Makefile.config's LIBRARY_DIRS