How to set the include path in a pybind11 project

Question

I have a large C++ library that I'm trying to expose using pybind11. I'm having trouble setting the include path correctly.

The project directories are structured as follows:

root
- sub-project-1
  -> C++ files that know nothing about python.
- sub-project-2
  -> more C++ files
  -> sub-sub-project
     -> even more C++ files
- sub-project-3
  -> et cetera
- Interfaces
  -> R
     -> R interface stuff
  -> python
     -> package_name
        -> setup.py
        -> pybind11_bindings.cpp 

The setup.py file currently looks like this. The structure is mainly copied from the pybind11 docs.

    from setuptools import setup, Extension
    from setuptools.command.build_ext import build_ext
    import sys
    import setuptools
    from glob import glob

    __version__ = '0.0.1'


    class get_pybind_include(object):
        """Helper class to determine the pybind11 include path

        The purpose of this class is to postpone importing pybind11
        until it is actually installed, so that the ``get_include()``
        method can be invoked. """

        def __init__(self, user=False):
            self.user = user

        def __str__(self):
            import pybind11
            return pybind11.get_include(self.user)

    # omitting long lists of files.
    my_sources = list_of_all_project_cpp_files + list_of_all_pybind11_binding_files

    ext_modules = [
        Extension(
            'Boom',
            sources=my_sources,
            include_dirs=[
                "../..",
                # Path to pybind11 headers
                get_pybind_include(),
                get_pybind_include(user=True)
            ],
            language='c++'
        ),
    ]
    # some other stuff...
    setup(
        name='package_name',
        version=__version__,
        author='Me',
        author_email='[email protected]',
        url='https://a/url/to/somewhere',
        description='description goes here.',
        ext_modules=ext_modules,
        install_requires=['pybind11>=2.3'],
        setup_requires=['pybind11>=2.3'],
        cmdclass={'build_ext': BuildExt},
        zip_safe=False,
    )

When I pip3 install ./package_name I get C++ compiler errors because library headers cannot be found. I have tried changing the 'include_dirs' argument to Extension to include several different layers of ../../.. to get to the top of the project directory, without success.

I would prefer to not copy the full library into the .../python/project_name directory, unless the experts here tell me I need to.

Answer 1

The missing include files needed to be included with the package by creating a MANIFEST.in file.

# This file lists all the header files to include with the distribution.
# See https://packaging.python.org/guides/using-manifest-in/ for syntax.

(commands using 'include' 'recursive-include' and 'graft' go here)