I am learning to develop with PyTorch as well as LibTorch. I have the following line in my ~/.bashrc for dynamic linking of libtorch libraries:
# libtorch linking path
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/user/.dev_libraries/libtorch/lib/
However, when this path is in LD_LIBRARY_PATH, importing torch in Python reports segmentation fault:
user@host:~$ $LD_LIBRARY_PATH
bash: /home/user/packages/embree-2.16.0.x86_64.linux/lib:/home/user/packages/embree-2.16.0.x86_64.linux/lib::/usr/local/lib/:/usr/local/cuda-11.1/lib64:/usr/local/lib/:/usr/local/cuda-11.1/lib64:/home/user/.dev_libraries/libtorch-cpu/libtorch/lib/: No such file or directory
user@host:~$ python
Python 3.8.10 (default, Jun 2 2021, 10:49:15)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
Segmentation fault (core dumped)
user@host:~$
As soon as I remove that path from the environment variable LD_LIBRARY_PATH, torch can be correctly imported in Python.
I am guessing the cause is that some shared libraries of PyTorch having the same names as the ones in LibTorch. Does this mean PyTorch and LibTorch cannot be installed simultaneously, or is my environment setting incorrect? I'd prefer not to reset LD_LIBRARY_PATH every time I switch between the two.
System specs:
I solved this problem.
Actually, there is no need to download c++ libtorch if you have already downloaded pytorch using pip. Assume that you are using conda, libtorch.so will be automatically installed in /root/miniconda3/envs/your_env/lib/python3.10/site-packages/torch/lib, alongside with many other libraries that c++ can use. Using this libtorch directly in your c++ code and cmake will cause no version confliction when importing it to python by pybind11, because they come from the same place. And pybind11 has no version issue, so any download method will be fine (git clone from source or pip) if you are using it.
set(TORCH_PATH "${CONDA_PATH}/lib/python3.10/site-packages/torch")
# Find pybind11 (from your own project)
add_subdirectory(extern/pybind11)
# Find Torch
find_package(Torch REQUIRED PATHS ${TORCH_PATH})
# Find torch_python lib
find_library(TORCH_PYTHON_LIBRARY torch_python "${TORCH_PATH}/lib")
if(TORCH_PYTHON_LIBRARY)
message(STATUS "Found TORCH_PYTHON_LIBRARY: ${TORCH_PYTHON_LIBRARY}")else()
message(WARNING "TORCH_PYTHON_LIBRARY not found!")
endif()
Here is part of my project's cmakelist.txt, may take this as an example.