how to solve conflict btw vcpkg and pybind11

System:

Mac OSX 15.0
apple clang 16.0.0
cmake 3.30.1

In recent versions of CMake the procedures for finding python have changed.

Suppose I have the following directory setup

extern/vcpkg/  # cloned from github and bootstrapped
extern/pybind11/  # cloned from github
CMakeLists.txt
main.cpp

With the following contents:

# CMakeLists.txt

# cmake_minimum_required(VERSION 3.26.0 FATAL_ERROR)
cmake_minimum_required(VERSION 3.27.0 FATAL_ERROR)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_EXPORT_COMPILE_COMMANDS 1)

# set (CMAKE_TOOLCHAIN_FILE ${CMAKE_SOURCE_DIR}/extern/vcpkg/scripts/buildsystems/vcpkg.cmake)

set(PROJECT mytest)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_SOURCE_DIR})
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_SOURCE_DIR})
project(${PROJECT})
add_subdirectory(extern/pybind11)

pybind11_add_module(${PROJECT} main.cpp)

// main.cpp

#include <pybind11/pybind11.h>
#include <iostream>

void say_hello(){ std::cout << "Hello world!" << std::endl; }

PYBIND11_MODULE(mytest, m) { m.def("say_hello", &say_hello); }

Then I build with:

rm -rf build || true && cmake -H. -Bbuild && cmake --build build --target all

Under this configuration, the module builds and runs just fine

If I uncomment the CAMKE_TOOLCHAIN_FILE line, then the build fails with the following message:

CMake Error at extern/vcpkg/scripts/buildsystems/vcpkg.cmake:859 (_find_package):
  By not providing "FindPythonInterp.cmake" in CMAKE_MODULE_PATH this project
  has asked CMake to find a package configuration file provided by
  "PythonInterp", but CMake did not find one.

  Could not find a package configuration file provided by "PythonInterp"
  (requested version 3.8) with any of the following names:

    PythonInterpConfig.cmake
    pythoninterp-config.cmake

  Add the installation prefix of "PythonInterp" to CMAKE_PREFIX_PATH or set
  "PythonInterp_DIR" to a directory containing one of the above files.  If
  "PythonInterp" provides a separate development package or SDK, be sure it
  has been installed.
Call Stack (most recent call first):
  extern/pybind11/tools/FindPythonLibsNew.cmake:114 (find_package)
  extern/vcpkg/scripts/buildsystems/vcpkg.cmake:859 (_find_package)
  extern/pybind11/tools/pybind11Tools.cmake:44 (find_package)
  extern/pybind11/tools/pybind11Common.cmake:200 (include)
  extern/pybind11/CMakeLists.txt:229 (include)

If I then revert the minimum required version of cmake, then the policy settings change and the build succeeds again but with the following warning from pybind:

CMake Warning (dev) at extern/pybind11/tools/FindPythonLibsNew.cmake:101 (message):
  Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
  are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
  the cmake_policy command to set the policy and suppress this warning, or
  preferably upgrade to using FindPython, either by calling it explicitly
  before pybind11, or by setting PYBIND11_FINDPYTHON ON before pybind11.
Call Stack (most recent call first):
  extern/vcpkg/scripts/buildsystems/vcpkg.cmake:859 (_find_package)
  extern/pybind11/tools/pybind11Tools.cmake:44 (find_package)
  extern/pybind11/tools/pybind11Common.cmake:200 (include)
  extern/pybind11/CMakeLists.txt:229 (include)

I don't really know what vcpkg is going to affect the settings here, so if someone knows, that would be useful to understand. Furthermore, I'm trying to understand if there is anything I can do to fix this problem so that I can use up to date policy settings

Solution

Pybind11 plays a dirty games with policy CMP0148, see e.g. the code at CMakeLists.txt:15. Probably, vcpkg modifies that policy for its own reasons, and that modification confuses Pybind11 (according to your error log, Pybind11 tries to use FindPythonInterp.cmake script which is marked as removed in CMake 3.27).

Since your minimal requirement to CMake version is above 3.12, where FindPython.cmake script has been added, you may safely tell Pybind11 to unconditionally use that script instead of deprecated ones. This can be achieved by setting

set(PYBIND11_FINDPYTHON ON)

before descending into Pybind11 code. (This is what the last warning message suggests you.)