Search code examples
pytorchgoogle-colaboratorykagglehuggingface-transformerstpu

OSError: libmkl_intel_lp64.so.1: cannot open shared object file: No such file or directory


I am trying to run a model on TPU as given in colab notebook. The model was working fine, but today I could not run the model.

I used the following code to install pytorch-xla.

VERSION = "nightly"  #@param ["1.5" , "20200325", "nightly"]
!curl https://raw.githubusercontent.com/pytorch/xla/master/contrib/scripts/env-setup.py -o pytorch-xla-env-setup.py
!python pytorch-xla-env-setup.py --version $VERSION

I try to install required libraries as below:

!pip install -U nlp
!pip install sentencepiece
!pip install numpy --upgrade

However, when I try the following

import nlp

It gives the following error:

OSError: libmkl_intel_lp64.so.1: cannot open shared object file: No such file or directory

I searched the error and I tried the followings, but still does not work. Any ideas how to fix it? Note: It was working a few days ago, however, today it is not.

!pip install mkl
#!export PATH="$PATH:/opt/intel/bin"
#!export LD_LIBRARY_PATH="$PATH:opt/intel/mkl/lib/intel64_lin/"
!export LID_LIBRAEY_PATH="$LID_LIBRARY_PATH:/opt/intel/mkl/lib/intel64_lin/"

Solution

  • import os 
    os.environ['LD_LIBRARY_PATH']='/usr/local/lib'
    
    !echo $LD_LIBRARY_PATH
    !sudo ln -s /usr/local/lib/libmkl_intel_lp64.so /usr/local/lib/libmkl_intel_lp64.so.1
    !sudo ln -s /usr/local/lib/libmkl_intel_thread.so /usr/local/lib/libmkl_intel_thread.so.1
    !sudo ln -s /usr/local/lib/libmkl_core.so /usr/local/lib/libmkl_core.so.1
    
    !ldconfig
    !ldd /usr/local/lib/python3.7/dist-packages/torch/lib/libtorch.so
    

    worked for me. We will also try to fix the problem internally.