Search code examples
c++makefilecompilationheader-filespybind11

Compile Pybind with header files in Makefile (not using cmake)


I am trying to compile a Pybind11 module in C++ which calls several header files (.h) on top. As I have a lot of header files, I decided to do a Makefile, which works without problem, EXCEPT for creating the target shared object file (s.o). I need this shared object file in order to be able to call the Pybind module in Python.

However, when compiling, I get:

g++ -shared -fPIC neat.o network.o nnode.o link.o trait.o gene.o innovation.o organism.o species.o genome.o population.o example.o -o example.so
/usr/bin/ld: example.o: relocation R_X86_64_PC32 against symbol `_ZTI3Pet' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
Makefile:2: recipe for target 'example.so' failed
make: *** [example.so] Error 1

My question is basically: What I am doing wrong when compiling the object files in order to create a target ?

Makefile

example.so: neat.o network.o nnode.o link.o trait.o gene.o innovation.o organism.o species.o genome.o population.o example.o
    g++ neat.o network.o nnode.o link.o trait.o gene.o innovation.o organism.o species.o genome.o population.o example.o -shared -o example.so

neat.o: neat.cpp neat.h
    g++ -c -O3 -Wall -fPIC neat.cpp

network.o: network.cpp network.h
    g++ -c -O3 -Wall -fPIC network.cpp

nnode.o: nnode.cpp nnode.h
    g++ -c -O3 -Wall -fPIC nnode.cpp

link.o: link.cpp link.h
    g++ -c -O3 -Wall -fPIC link.cpp

trait.o: trait.cpp trait.h
    g++ -c -O3 -Wall -fPIC trait.cpp

gene.o: gene.cpp gene.h
    g++ -c -O3 -Wall -fPIC gene.cpp

innovation.o: innovation.cpp innovation.h
    g++ -c -O3 -Wall -fPIC innovation.cpp

organism.o: organism.cpp organism.h genome.h genome.cpp species.h species.cpp
    g++ -c -O3 -Wall -fPIC organism.cpp

species.o: species.h species.cpp organism.cpp organism.h genome.h genome.cpp
    g++ -c -O3 -Wall -fPIC species.cpp

genome.o: genome.cpp genome.h
    g++ -c -O3 -Wall -fPIC genome.cpp

population.o: population.cpp population.h organism.h
    g++ -c -O3 -Wall -fPIC population.cpp

experiments.o: experiments.cpp experiments.h
    g++ -c -O3 -Wall -fPIC experiments.cpp

example.o:
    g++ -O3 -Wall -std=c++11 -fopenmp -I -fPIC `python3 -m pybind11 --includes` -c example.cpp

clean:
    rm *.o *.so

example.cpp

#include <pybind11/pybind11.h>
#include <iostream>
#include <string>
#include "population.h"

namespace py = pybind11;

int create_neat(){
  Population *the_pop=0;
  return 0;
}
PYBIND11_MODULE(example, m){

  m.def("create_neat", &create_neat, "create a pop object");

}

Solution

  • After several hours of reading, trying, and also thanks to the hints of MadScientist, I got it working but the reasons why are still unclear. I just changed the order of the prerequisites in the compilation of the target, by putting the -shared -o example.so in front and not at the end of the line, and it worked. The whole file is as follows:

    example.so: neat.o network.o nnode.o link.o trait.o gene.o innovation.o organism.o species.o genome.o population.o example.o
        g++ -shared -o example.so neat.o network.o nnode.o link.o trait.o gene.o innovation.o organism.o species.o genome.o population.o example.o
    
    neat.o: neat.cpp neat.h
        g++ -c -O3 -Wall -fPIC neat.cpp
    
    network.o: network.cpp network.h
        g++ -c -O3 -Wall -fPIC network.cpp
    
    nnode.o: nnode.cpp nnode.h
        g++ -c -O3 -Wall -fPIC nnode.cpp
    
    link.o: link.cpp link.h
        g++ -c -O3 -Wall -fPIC link.cpp
    
    trait.o: trait.cpp trait.h
        g++ -c -O3 -Wall -fPIC trait.cpp
    
    gene.o: gene.cpp gene.h
        g++ -c -O3 -Wall -fPIC gene.cpp
    
    innovation.o: innovation.cpp innovation.h
        g++ -c -O3 -Wall -fPIC innovation.cpp
    
    organism.o: organism.cpp organism.h genome.h genome.cpp species.h species.cpp
        g++ -c -O3 -Wall -fPIC organism.cpp
    
    species.o: species.h species.cpp organism.cpp organism.h genome.h genome.cpp
        g++ -c -O3 -Wall -fPIC species.cpp
    
    genome.o: genome.cpp genome.h
        g++ -c -O3 -Wall -fPIC genome.cpp
    
    population.o: population.cpp population.h organism.h
        g++ -c -O3 -Wall -fPIC population.cpp
    
    experiments.o: experiments.cpp experiments.h
        g++ -c -O3 -Wall -fPIC experiments.cpp
    
    example.o:
        g++ -O3 -Wall -std=c++11 -fopenmp `python3 -m pybind11 --includes` -fPIC -c example.cpp
    
    clean:
        rm *.o *.so