Search code examples
pythonsetuptoolssetup.py

Why does distutils.core.setup build shared extensions in 2 compiler invocations?


distutils.core.setup, when told to build a shared extension module from a single C source file, does it in two steps:

In the first, it invokes the compiler and builds an object file with -fPIC. A shortened commandline of this step could look something like this (at least on a unix system):

gcc -pthread -I/usr/include/python3.8 -O3 -fPIC -c ./SOURCE.c -o build/.../ext.o

In the second, it feeds the aforementioned object file to the compiler again, this time passing an additional -shared compiler flag, and produces a shared .so library file.

gcc -pthread -O1 -shared ./build/.../ext.o -o ./build/.../ext.so


This could surely be achieved in one singular step (I managed to manually reproduce a working extension using one compiler invocation) so why is it done in two? This is a pretty niche implementation detail but I'm very curious.


Solution

  • The 1st run is compilation, it compiles SOURCE.c to ext.o. The 2nd run is linking, it links ext.o with the standard libraries and produces the final ext.so.

    Why there are 2 separate runs is hard to say. Perhaps because distutils/setuptools are designed to be universal and use different compilers, not only gcc. See the list of supported compilers in the source code; look at *compiler.py.