caffe import error: even after installing it successfully, on ubuntu 18.04

Question

I'm getting caffe import error even after installing it successfully using the command sudo apt install caffe-cpu. I was able to find caffe file at /usr/lib/python3/dist-packages/caffe (Path was added to PYTHONPATH). All requirements mentioned in the requirements.txt file of caffe directory was also installed.

I'm using Ubuntu 18.04 LTS, Python3.

Could anyone help me with this error?

import caffe        

Traceback (most recent call last):
      File "6_reconstruct_alphabet_image.py", line 17, in <module>
        import caffe
      File "/usr/lib/python3/dist-packages/caffe/__init__.py", line 1, in <module>
        from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
      File "/usr/lib/python3/dist-packages/caffe/pycaffe.py", line 15, in <module>
        import caffe.io
      File "/usr/lib/python3/dist-packages/caffe/io.py", line 2, in <module>
        import skimage.io
      File "/usr/lib/python3/dist-packages/skimage/__init__.py", line 158, in <module>
        from .util.dtype import *
      File "/usr/lib/python3/dist-packages/skimage/util/__init__.py", line 7, in <module>
        from .arraycrop import crop
      File "/usr/lib/python3/dist-packages/skimage/util/arraycrop.py", line 8, in <module>
        from numpy.lib.arraypad import _validate_lengths
    ImportError: cannot import name '_validate_lengths'

Answer 1

Problem solved: The error came up because caffe build wasn't done successfully. I recommend not to go up with the sudo apt install caffe-cpu command (Which is mentioned in the official caffe installation guide for Ubuntu); because it will end up in the error as above. It's better to install from the source.

Let me give step by step guidance to install caffe successfully in Ubuntu 18.04 LTS:

1] sudo apt-get install -y --no-install-recommends libboost-all-dev

2] sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev libhdf5-serial-dev \ libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler

3] git clone https://github.com/BVLC/caffe cd caffe cp Makefile.config.example Makefile.config

4] sudo pip install scikit-image protobuf cd python for req in $(cat requirements.txt); do sudo pip install $req; done

5] Modify the Makefile.config file: Uncomment the line CPU_ONLY := 1, and the line OPENCV_VERSION := 3.

6] Find LIBRARIES line in Makefile and change it to as follows:

LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5 \ opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs

7] make all

Now you could get some error like this:

CXX src/caffe/net.cpp src/caffe/net.cpp:8:18: fatal error: hdf5.h: No such file or directory compilation terminated. Makefile:575: recipe for target '.build_release/src/caffe/net.o' failed make: *** [.build_release/src/caffe/net.o] Error 1

To solve this error follow step 8.

8] install libhdf5-dev open Makefile.config, locate line containing LIBRARY_DIRS and append /usr/lib /x86_64-linux-gnu/hdf5/serial locate INCLUDE_DIRS and append /usr/include/hdf5/serial/ (per this SO answer) rerun make all

9] make test

10] make runtest

11] make pycaffe

Now you could get some error like this:

CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp python/caffe/_caffe.cpp:10:31: fatal error: numpy/arrayobject.h: No such file or directory compilation terminated. Makefile:501: recipe for target 'python/caffe/_caffe.so' failed make: *** [python/caffe/_caffe.so] Error 1

To solve this error follow step 12.

12] Find PYTHON_INCLUDE line in Makefile.config and do the changes as follows:

`PYTHON_INCLUDE := /usr/include/python2.7 \
/usr/local/lib/python2.7/dist-packages/numpy/core/include`

13] Add the module directory to our $PYTHONPATH by adding this line to the end of ~/.bashrc file:

`sudo vim ~/.bashrc`
`export PYTHONPATH=$HOME/Downloads/caffe/python:$PYTHONPATH`
`source ~/.bashrc'

14] Done.