Search code examples
pytorchlinker-errorslibtorch

Pytorch Tensor::data_ptr<long long>() not working on Linux

I cannot link my program to pytorch under Linux, get the following error:

/tmp/ccbgkLx2.o: In function `long long* at::Tensor::data<long long>() const':
test.cpp:(.text._ZNK2at6Tensor4dataIxEEPT_v[_ZNK2at6Tensor4dataIxEEPT_v]+0x14): undefined reference to `long long* at::Tensor::data_ptr<long long>() const'

I am building a very simple minimal example:

#include "torch/script.h"
#include <iostream>

int main() {
    auto options = torch::TensorOptions().dtype(torch::kInt64);
    torch::NoGradGuard no_grad;
    auto T = torch::zeros(20, options).view({ 10, 2 });
    long long *data = (long long *)T.data<long long>();
    data[0] = 1;
    return 0;
}

The command used to build it:

g++ -w -std=c++17 -o test-torch test.cpp -D_GLIBCXX_USE_CXX11_ABI=1 -Wl,--whole-archive -ldl -lpthread -Wl,--no-whole-archive -I../libtorch/include -L../libtorch/lib -ltorch -ltorch_cpu -lc10 -Wl,-rpath,../libtorch/lib

Pytorch has been downloaded from the link https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-1.7.0%2Bcpu.zip and unzipped (so I have the libtorch folder next to the folder with test.cpp).

Any ideas how to solve this problem? Same program works just fine under Visual C++.

P.S. I know pytorch is kind of designed for cmake, but I have zero experience with cmake and no desire to write a cmake-based build system for my app. Also, the examples they give are seemingly supposed to only work if pytorch is "installed" in the system. So I cannot just download the .zip with libs? And if I "install" it (e.g. from sources or in whatever other way) on an AVX512 system, will the binary I link to it and distribute to end-users work on non-AVX512? The documentation is completely incomprehensible for newbies.

UPDATE: I tried to do this via CMake following the tutorial https://pytorch.org/cppdocs/installing.html and got exactly the same error. Specifically, I renamed my directory to example-app and the source file to example-app.cpp. Then I created CMakeLists.txt in this directory with the following contents:

cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
project(example-app)

find_package(Torch REQUIRED)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")

add_executable(example-app example-app.cpp)
target_link_libraries(example-app "${TORCH_LIBRARIES}")
set_property(TARGET example-app PROPERTY CXX_STANDARD 14)

Then

mkdir build
cd build
cmake -DCMAKE_PREFIX_PATH=../../libtorch ..
cmake --build . --config Release

And here's the output:

CMakeFiles/example-app.dir/example-app.cpp.o: In function `long long* at::Tensor::data<long long>() const':
example-app.cpp:(.text._ZNK2at6Tensor4dataIxEEPT_v[_ZNK2at6Tensor4dataIxEEPT_v]+0x14): undefined reference to `long long* at::Tensor::data_ptr<long long>() const'

Makes me think, maybe I forgot to include some header or define some variable? Oh, this is all on Mint 19.2 (equivalent to Ubuntu 18.04), g++ version is 7.5.0, glibc is 2.27. Compiling with g++-8 gives the same result.

Solution

  • This is not a cmake-related error, it's just how the library was implemented. I do not know why, but it appears that the specialization of T* at::Tensor::data<T> const with T = long long was forgotten/omitted.

    If you want to get your signed 64-bits pointer, you can still get it with int64_t:

    auto data = T.data<int64_t>();

    It's good practice to use these types for which the size is explicit in general, in order to avoid compatibility issues.