How to properly satisfy all linker dependencies in following pybind11 project?

Question

I am porting a large library to pybind11 for python interface. However I have stuck where I seemingly am either getting multiple declaration error at linker stage, or during python import I get symbol not found error. The simplified project structure is as given below

CMake file

cmake_minimum_required(VERSION 3.5)

# set the project name
project(tmp VERSION 1.0)

find_package(EnvModules REQUIRED)
env_module(load python39)


find_package(PythonInterp REQUIRED)
include_directories(${PYTHON_INCLUDE_DIRS})
include_directories(pybind11/include)

add_subdirectory(pybind11)
pybind11_add_module(TMP py_modules.cpp
                        SubConfiguration.cpp)
# pybind11_add_module(TMP py_modules.cpp)

Source files: py_modules.cpp

#include "pybind11/pybind11.h"
#include "calculateStress.h"

namespace py = pybind11;

PYBIND11_MODULE(TMP, m){
    py::class_<Stencil>(m, "Stencil")
            .def(py::init<double &>());
    py::class_<SubConfiguration>(m, "SubConfiguration")
            .def(py::init<Stencil &>());
}

2. calculateStress.h

#ifndef CALCULATESTRESS_H_
#define CALCULATESTRESS_H_

#include "SubConfiguration.h"

#endif /* CALCULATESTRESS_H_ */

3. SubConfiguration.h

#ifndef SRC_SUBCONFIGURATION_H_
#define SRC_SUBCONFIGURATION_H_

#include "Stencil.h"

class SubConfiguration
{
public:
    double& parent;
    SubConfiguration(Stencil& stencil);
};

#endif /* SRC_SUBCONFIGURATION_H_ */

4. SubConfiguration.cpp

#include "SubConfiguration.h"

SubConfiguration::SubConfiguration(Stencil& stencil) :
    parent(stencil.parent)
{}

5. Stencil.h

#ifndef SRC_STENCIL_H_
#define SRC_STENCIL_H_

class Stencil {
public:
    double parent;
    Stencil(double&);
};

Stencil::Stencil(double& parent) : parent(parent) { }

#endif /* SRC_STENCIL_H_ */

I think all import guards are properly set.

Now when I give pybind11_add_module(TMP py_modules.cpp) compile option, I get the TMP module but while importing I get error undefined symbol: _ZN16SubConfigurationC1ER7Stencil

Whereas with pybind11_add_module(TMP py_modules.cpp SubConfiguration.cpp) I get following error

/usr/bin/ld: CMakeFiles/TMP.dir/SubConfiguration.cpp.o (symbol from plugin): in function `Stencil::Stencil(double&)':
(.text+0x0): multiple definition of `Stencil::Stencil(double&)'; CMakeFiles/TMP.dir/py_modules.cpp.o (symbol from plugin):(.text+0x0): first defined here
/usr/bin/ld: CMakeFiles/TMP.dir/SubConfiguration.cpp.o (symbol from plugin): in function `Stencil::Stencil(double&)':
(.text+0x0): multiple definition of `Stencil::Stencil(double&)'; CMakeFiles/TMP.dir/py_modules.cpp.o (symbol from plugin):(.text+0x0): first defined here
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/TMP.dir/build.make:99: TMP.cpython-39-x86_64-linux-gnu.so] Error 1
make[1]: *** [CMakeFiles/Makefile2:96: CMakeFiles/TMP.dir/all] Error 2
make: *** [Makefile:84: all] Error 2

How do I properly set it up?

Answer 1

The problem is that Stencil.h gets included by both py_modules.cpp and SubConfiguration.cpp. The include guards will only protect against double includes from a single compilation unit (cpp file).

Remove the implementation of the Stencil constructor from Stencil.h and move it into a new file Stencil.cpp as

#include "Stencil.h"
Stencil::Stencil(double& parent) : parent(parent) { }

Then add the new file to your module

...
add_subdirectory(pybind11)
pybind11_add_module(TMP py_modules.cpp SubConfiguration.cpp Stencil.cpp )

UPDATE (to reflect comments)

If you have a templated class, you do not need necessarily to include the implementation in the header. You can use explicit template instantiation as in this example:

In the header doit.h

template< typename T >
T addsome( T t );

In the body doit.cpp

#include "doit.h"

// Add explicit template instantiations for the types you care
// #include <doit.h>
template<> 
int addsome<int>( int t ) {
    return t+1;
}
template<> 
double addsome<double>( double t ) {
    return t+2;
}

In main.cpp or where you use it

int main() {
    int res = addsome<int>( 1 ); // This will link against the compiled body doit.cpp
}