Could NOT find Boost (missing: python37) (found version "1.80.0")

Question

Not able to install vowpalwabbit on Apple M1 MacBook Pro with Python 3.7.16. Getting the following error instead. I have tried the following but was not able to import the package successfully.

1. Install dependencies with brew install cmake and brew install boost
2. Installed the package directly like brew install vowpal-wabbit
3. Installed an older version of Boost with brew install [email protected]

User@MacBook ~ % python -m pip install vowpalwabbit
Keyring is skipped due to an exception: 'EntryPoints' object has no attribute 'get'
Collecting vowpalwabbit
  Using cached vowpalwabbit-9.6.0.tar.gz (16.8 MB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: vowpalwabbit
  Building wheel for vowpalwabbit (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [80 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-12.5-arm64-3.7
      creating build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/pyvw.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/version.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/__init__.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/sklearn_vw.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/sklearn.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/__main__.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/dftovw.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      running egg_info
      writing python/vowpalwabbit.egg-info/PKG-INFO
      writing dependency_links to python/vowpalwabbit.egg-info/dependency_links.txt
      writing top-level names to python/vowpalwabbit.egg-info/top_level.txt
      reading manifest file 'python/vowpalwabbit.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no files found matching 'CMakeSettings.json'
      warning: no files found matching 'python/*.bat'
      warning: no files found matching '*' under directory 'explore'
      warning: no files found matching 'python/*.dll' under directory 'python/bin'
      warning: no previously-included files matching '*.o' found anywhere in distribution
      warning: no previously-included files matching '*.exe' found anywhere in distribution
      warning: no previously-included files matching '*.pyc' found anywhere in distribution
      warning: no previously-included files matching '*' found under directory 'vowpalwabbit/.nuget'
      warning: no previously-included files matching '*' found under directory 'ext_libs/boost_math/doc'
      warning: no previously-included files matching '*' found under directory 'ext_libs/boost_math/test'
      writing manifest file 'python/vowpalwabbit.egg-info/SOURCES.txt'
      copying python/vowpalwabbit/conda_install.sh -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      running build_ext
      creating build/temp.macosx-12.5-arm64-3.7
      cmake -DCMAKE_BUILD_TYPE=Release -DPY_VERSION=3.7 -DBUILD_PYTHON=On -DBUILD_TESTING=Off -DWARNINGS=Off -DVW_PYTHON_SHARED_LIB_SUFFIX=.cpython-37m-darwin.so -DBoost_NO_BOOST_CMAKE=ON -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/private/var/folders/0q/gqwnktxx441c9kt53hjnxwj40000gp/T/pip-install-v6s_dtz3/vowpalwabbit_7f17cef39a4f41268477085c998014a7/build/lib.macosx-12.5-arm64-3.7 /private/var/folders/0q/gqwnktxx441c9kt53hjnxwj40000gp/T/pip-install-v6s_dtz3/vowpalwabbit_7f17cef39a4f41268477085c998014a7
      -- VowpalWabbit Version: 9.6.0
      -- The C compiler identification is AppleClang 14.0.0.14000029
      -- The CXX compiler identification is AppleClang 14.0.0.14000029
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Using C++ standard: 11
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
      -- Found Threads: TRUE
      -- Module support is disabled.
      -- Version: 9.1.0
      -- Build type: Release
      -- CXX_STANDARD: 11
      -- Performing Test has_std_11_flag
      -- Performing Test has_std_11_flag - Success
      -- Performing Test has_std_0x_flag
      -- Performing Test has_std_0x_flag - Success
      -- Required features: cxx_variadic_templates
      -- Performing Test HAS_NULLPTR_WARNING
      -- Performing Test HAS_NULLPTR_WARNING - Success
      -- Build spdlog: 1.10.0
      -- Build type: Release
      -- Boost.Math: standalone mode ON
      -- Found ZLIB: /Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk/usr/lib/libz.tbd (found version "1.2.11")
      -- Python 3.7 will be used for VowpalWabbit Python bindings
      -- Found Boost: /opt/homebrew/include (found version "1.80.0")
      CMake Error at /Users/User/.pyenv/versions/3.7.16/lib/python3.7/site-packages/cmake/data/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
        Could NOT find Boost (missing: python37) (found version "1.80.0")
      Call Stack (most recent call first):
        /Users/User/.pyenv/versions/3.7.16/lib/python3.7/site-packages/cmake/data/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
        /Users/User/.pyenv/versions/3.7.16/lib/python3.7/site-packages/cmake/data/share/cmake-3.25/Modules/FindBoost.cmake:2376 (find_package_handle_standard_args)
        python/CMakeLists.txt:28 (find_package)


      -- Configuring incomplete, errors occurred!
      See also "/private/var/folders/0q/gqwnktxx441c9kt53hjnxwj40000gp/T/pip-install-v6s_dtz3/vowpalwabbit_7f17cef39a4f41268477085c998014a7/build/temp.macosx-12.5-arm64-3.7/CMakeFiles/CMakeOutput.log".
      See also "/private/var/folders/0q/gqwnktxx441c9kt53hjnxwj40000gp/T/pip-install-v6s_dtz3/vowpalwabbit_7f17cef39a4f41268477085c998014a7/build/temp.macosx-12.5-arm64-3.7/CMakeFiles/CMakeError.log".
      error: command 'cmake' failed with exit status 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for vowpalwabbit
  Running setup.py clean for vowpalwabbit
Failed to build vowpalwabbit
Installing collected packages: vowpalwabbit
  Running setup.py install for vowpalwabbit ... error
  error: subprocess-exited-with-error

  × Running setup.py install for vowpalwabbit did not run successfully.
  │ exit code: 1
  ╰─> [80 lines of output]
      running install
      running build
      running build_py
      creating build
      creating build/lib.macosx-12.5-arm64-3.7
      creating build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/pyvw.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/version.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/__init__.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/sklearn_vw.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/sklearn.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/__main__.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      copying python/vowpalwabbit/dftovw.py -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      running egg_info
      writing python/vowpalwabbit.egg-info/PKG-INFO
      writing dependency_links to python/vowpalwabbit.egg-info/dependency_links.txt
      writing top-level names to python/vowpalwabbit.egg-info/top_level.txt
      reading manifest file 'python/vowpalwabbit.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no files found matching 'CMakeSettings.json'
      warning: no files found matching 'python/*.bat'
      warning: no files found matching '*' under directory 'explore'
      warning: no files found matching 'python/*.dll' under directory 'python/bin'
      warning: no previously-included files matching '*.o' found anywhere in distribution
      warning: no previously-included files matching '*.exe' found anywhere in distribution
      warning: no previously-included files matching '*.pyc' found anywhere in distribution
      warning: no previously-included files matching '*' found under directory 'vowpalwabbit/.nuget'
      warning: no previously-included files matching '*' found under directory 'ext_libs/boost_math/doc'
      warning: no previously-included files matching '*' found under directory 'ext_libs/boost_math/test'
      writing manifest file 'python/vowpalwabbit.egg-info/SOURCES.txt'
      copying python/vowpalwabbit/conda_install.sh -> build/lib.macosx-12.5-arm64-3.7/vowpalwabbit
      running build_ext
      creating build/temp.macosx-12.5-arm64-3.7
      cmake -DCMAKE_BUILD_TYPE=Release -DPY_VERSION=3.7 -DBUILD_PYTHON=On -DBUILD_TESTING=Off -DWARNINGS=Off -DVW_PYTHON_SHARED_LIB_SUFFIX=.cpython-37m-darwin.so -DBoost_NO_BOOST_CMAKE=ON -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/private/var/folders/0q/gqwnktxx441c9kt53hjnxwj40000gp/T/pip-install-v6s_dtz3/vowpalwabbit_7f17cef39a4f41268477085c998014a7/build/lib.macosx-12.5-arm64-3.7 /private/var/folders/0q/gqwnktxx441c9kt53hjnxwj40000gp/T/pip-install-v6s_dtz3/vowpalwabbit_7f17cef39a4f41268477085c998014a7
      -- VowpalWabbit Version: 9.6.0
      -- The C compiler identification is AppleClang 14.0.0.14000029
      -- The CXX compiler identification is AppleClang 14.0.0.14000029
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Using C++ standard: 11
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
      -- Found Threads: TRUE
      -- Module support is disabled.
      -- Version: 9.1.0
      -- Build type: Release
      -- CXX_STANDARD: 11
      -- Performing Test has_std_11_flag
      -- Performing Test has_std_11_flag - Success
      -- Performing Test has_std_0x_flag
      -- Performing Test has_std_0x_flag - Success
      -- Required features: cxx_variadic_templates
      -- Performing Test HAS_NULLPTR_WARNING
      -- Performing Test HAS_NULLPTR_WARNING - Success
      -- Build spdlog: 1.10.0
      -- Build type: Release
      -- Boost.Math: standalone mode ON
      -- Found ZLIB: /Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk/usr/lib/libz.tbd (found version "1.2.11")
      -- Python 3.7 will be used for VowpalWabbit Python bindings
      -- Found Boost: /opt/homebrew/include (found version "1.80.0")
      CMake Error at /Users/User/.pyenv/versions/3.7.16/lib/python3.7/site-packages/cmake/data/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
        Could NOT find Boost (missing: python37) (found version "1.80.0")
      Call Stack (most recent call first):
        /Users/User/.pyenv/versions/3.7.16/lib/python3.7/site-packages/cmake/data/share/cmake-3.25/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
        /Users/User/.pyenv/versions/3.7.16/lib/python3.7/site-packages/cmake/data/share/cmake-3.25/Modules/FindBoost.cmake:2376 (find_package_handle_standard_args)
        python/CMakeLists.txt:28 (find_package)


      -- Configuring incomplete, errors occurred!
      See also "/private/var/folders/0q/gqwnktxx441c9kt53hjnxwj40000gp/T/pip-install-v6s_dtz3/vowpalwabbit_7f17cef39a4f41268477085c998014a7/build/temp.macosx-12.5-arm64-3.7/CMakeFiles/CMakeOutput.log".
      See also "/private/var/folders/0q/gqwnktxx441c9kt53hjnxwj40000gp/T/pip-install-v6s_dtz3/vowpalwabbit_7f17cef39a4f41268477085c998014a7/build/temp.macosx-12.5-arm64-3.7/CMakeFiles/CMakeError.log".
      error: command 'cmake' failed with exit status 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> vowpalwabbit

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Answer 1

It couldn't find the Boost python component specifically. You could try installing the brew package for that component, however, Boost python depends on a specific python version so you'd be relying on homebrew to provide the right one and it might not work. Based on the homebrew package page that one is currently targeting Python 3.11.

The Vowpal Wabbit has binary wheels for MacOS arm for 3.8, 3.9 and 3.10 (See here to see currently supported versions). So, if you are able to use one of those versions that would be my recommendation.