What does the cmake variable Torch_DIR do? How do I find out what directory to hint for find_package(Torch)?

Question

In an official tutorial showing CMake for a project to depend on libtorch, we have this CMakeLists.txt:

cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
project(example-app)

find_package(Torch REQUIRED)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")

add_executable(example-app example-app.cpp)
target_link_libraries(example-app "${TORCH_LIBRARIES}")
set_property(TARGET example-app PROPERTY CXX_STANDARD 14)

# The following code block is suggested to be used on Windows.
# According to https://github.com/pytorch/pytorch/issues/25457,
# the DLLs need to be copied to avoid memory errors.
if (MSVC)
  file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
  add_custom_command(TARGET example-app
                     POST_BUILD
                     COMMAND ${CMAKE_COMMAND} -E copy_if_different
                     ${TORCH_DLLS}
                     $<TARGET_FILE_DIR:example-app>)
endif (MSVC)

Now, the documentation is old and does not work for many people, as referenced by this github issue:https://github.com/pytorch/pytorch/issues/12449

The solution that was suggested (and works for me) was to set an additional variable:

set(Torch_DIR "${CMAKE_SOURCE_DIR}/external/libtorch/share/cmake/Torch")

In this directory there are some .cmake files with comments at the top that says "Finds the torch library".

What I don't understand is why this variable has to be set in addition to the call to cmake -DCMAKE_PREFIX_PATH=/absolute/path/to/libtorch .., which is what the libtorch documentation instructs you to do.

I thought the CMAKE_PREFIX_PATH is the variable set in order to use find_package()

My question is: Why do you need both for it to work?

Answer 1

PyTorch apparently provides utilities for getting what the CMake prefix path to use is: run the following python -c 'import torch;print(torch.utils.cmake_prefix_path)'. Whatever you get from that is what you should either put in your CMAKE_PREFIX_PATH, or set as Torch_DIR. See also this thread on the CMake Discourse.

PyTorch uses find-package config scripts (not find module scripts) (see also its cmake/TorchConfig.cmake.in file). You shouldn't need to both put a find-package script's containing directory in CMAKE_PREFIX_PATH and in <PACKAGE-NAME>_DIR. Only one is necessary (and you can choose which one you want to use).

For how CMAKE_PREFIX_PATH comes into play, see step 2 in https://cmake.org/cmake/help/latest/command/find_package.html#config-mode-search-procedure. Note though that CMAKE_PREFIX_PATH has implications/uses also for find_program, find_library, find_file, and find_path.

The <PACKAGE-NAME>_DIR variable is actually a CMake cache variable that's supposed to store the result of running the find search procedure, so setting it directly is just short-circuiting that search and providing the result right away (at least- that's my understanding, which might be wrong). From the docs:

Config mode search attempts to locate a configuration file provided by the package to be found. A cache entry called <PackageName>_DIR is created to hold the directory containing the file.