Search code examples
pytorchanacondacondaanaconda3

libtorch_cpu.so: undefined symbol: iJIT_IsProfilingActive

I have created this Conda environment:

conda env create -f environment.yml

The environment.yml file:

name: deep3d_pytorch
channels:
  - pytorch
  - conda-forge
  - defaults
dependencies:
  - python=3.6
  - pytorch=1.6.0
  - torchvision=0.7.0
  - numpy=1.18.1
  - scikit-image=0.16.2
  - scipy=1.4.1
  - pillow=6.2.1
  - pip
  - ipython=7.13.0
  - yaml=0.1.7
  - pip:
    - matplotlib==2.2.5
    - opencv-python==3.4.9.33
    - tensorboard==1.15.0
    - tensorflow==1.15.0
    - kornia==0.5.5
    - dominate==2.6.0
    - trimesh==3.9.20

I activate the Conda environment. But even a simple statement like python -c "import torch; print(torch.__version__)" to get the PyTorch version throws the undefined symbol error:

(deep3d_pytorch) m3@i7:~/repos/Deep3DFaceRecon_pytorch> python -c "import torch; print(torch.__version__)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/m3/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/__init__.py", line 189, in <module>
    from torch._C import *
ImportError: /home/m3/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_IsProfilingActive

I believe the PyTorch installed by Conda is broken. But the Conda logs are all fine. Does anyone have a clue or hint? I'm receiving the undefined symbol error on both my local machine and on Google Colab.

Update: minimal env

Even a minimal Environment like below, would throw similar errors:

conda create -n minimal_pytorch python=3.6 pytorch torchvision torchaudio -c pytorch
source activate minimal_pytorch && python -c "import torch; print(torch.__version__)"

A similar undefined symbol error is thrown:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/envs/minimal_pytorch/lib/python3.6/site-packages/torch/__init__.py", line 197, in <module>
    from torch._C import *  # noqa: F403
ImportError: /usr/local/envs/minimal_pytorch/lib/python3.6/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent

Python version

When Python version is omitted while creating the environment:

conda create -n minimal_pytorch python pytorch torchvision torchaudio -c pytorch

The error is resolved:

source activate minimal_pytorch && python -c "import torch; print(torch.__version__)"

PyTorch version is received without any error:

2.2.2
Solution

  • To show how it was solved, I post all our code here. We were trying to test this repository:

    https://github.com/sicxu/Deep3DFaceRecon_pytorch

    Our final approach on Google Colab that works just fine is:

    # Step 1: Press runtime on the top > Change runtime type > Select T4 GPU

# Step 2: Run this shell

!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
!bash miniconda.sh -b -u -p /usr/local
!rm miniconda.sh
!conda update -y conda

%cd /content/
!rm -rf Deep3DFaceRecon_pytorch
!git clone https://github.com/sicxu/Deep3DFaceRecon_pytorch.git
%cd Deep3DFaceRecon_pytorch
!git clone https://github.com/NVlabs/nvdiffrast
!git clone https://github.com/deepinsight/insightface.git
!cp -r ./insightface/recognition/arcface_torch ./models/

# Step 3: Mount drive and copy the files mounted in google drive:

!mkdir /content/Deep3DFaceRecon_pytorch/checkpoints
!cp -r /content/drive/MyDrive/Deep3D/facerecon_20230425 /content/Deep3DFaceRecon_pytorch/checkpoints/facerecon_20230425
!cp /content/drive/MyDrive/Deep3D/01_MorphableModel.mat /content/Deep3DFaceRecon_pytorch/BFM/
!cp /content/drive/MyDrive/Deep3D/Exp_Pca.bin /content/Deep3DFaceRecon_pytorch/BFM/

# Step 4: Install the graphics related stuff

!sudo apt-get install libegl1-mesa-dev
!nvidia-smi
!sudo apt-get install libnvidia-gl-535

# Step 5: Run this shell to install everything

%%shell
eval "$(conda shell.bash hook)"
conda create --name deep3d_pytorch python=3.6 -y
conda activate deep3d_pytorch
conda config --env --add channels pytorch
conda config --env --add channels conda-forge
conda config --env --add channels defaults
conda install pytorch==1.6.0 torchvision==0.7.0 cudatoolkit=10.2 -c pytorch -y
conda install numpy scikit-image=0.16.2 scipy=1.4.1 pillow=6.2.1 pip ipython=7.13.0 yaml=0.1.7 -y
pip install matplotlib==2.2.5 opencv-python==3.4.9.33 tensorboard==1.15.0 tensorflow==1.15.0 kornia==0.5.5 dominate==2.6.0 trimesh==3.9.20
pip install ./nvdiffrast/.

# Step 6: Run this shell to test the program

%%shell
eval "$(conda shell.bash hook)"
conda activate deep3d_pytorch
python test.py --name=facerecon_20230425 --epoch=20 --img_folder=./datasets/examples

# Step 7: Copy test results to Google Drive

!ls /content/Deep3DFaceRecon_pytorch/checkpoints/facerecon_20230425/results/examples/epoch_20_000000/
!cp /content/Deep3DFaceRecon_pytorch/checkpoints/facerecon_20230425/results/examples/epoch_20_000000/* /content/drive/MyDrive/Deep3D/results/

# Step 8: prepare custom images along with their facial landmarks

!cp -r /content/drive/MyDrive/Deep3D/custom_images /content/Deep3DFaceRecon_pytorch/
%cd /content/Deep3DFaceRecon_pytorch/custom_images/
!python -m venv virtual_env
!source virtual_env/bin/activate && which pip
!source virtual_env/bin/activate && pip install mtcnn
!source virtual_env/bin/activate && pip install tensorflow
!source virtual_env/bin/activate && which python
!source virtual_env/bin/activate && python facial_landmarks.py # This Python script source code can be found here: https://github.com/sicxu/Deep3DFaceRecon_pytorch/issues/85#issuecomment-2069302718
%cd /content/Deep3DFaceRecon_pytorch/

# Step 9: run with custom images

%%shell
eval "$(conda shell.bash hook)"
conda activate deep3d_pytorch
python test.py --name=facerecon_20230425 --epoch=20 --img_folder=./custom_images

# Step 10: copy the results back to Google Drive

!ls /content/Deep3DFaceRecon_pytorch/checkpoints/facerecon_20230425/results/custom_images/epoch_20_000000/
!cp /content/Deep3DFaceRecon_pytorch/checkpoints/facerecon_20230425/results/custom_images/epoch_20_000000/* /content/drive/MyDrive/Deep3D/results/

    It's described here too: https://github.com/conda/conda/issues/13812#issuecomment-2071445372