Search code examples
makefilegnu-makeautotoolsautoconfautomake

Running a build with multiple jobs (e. g.: -j8) seems to produce a race with Autotools-generated Makefiles

Consider I have a simple Autotools project consisting of just one source file (e. g.: amhello).

Now, for some reason, despite there's a direct dependency between all and check targets in the generated Makefile, there seems to also be a race, so running

make -j8 all check

is likely to result in a build error:

make  all-recursive
Making check in src
make[1]: Entering directory '/tmp/tmp.2E2WN3dwFg/build'
make[1]: Entering directory '/tmp/tmp.2E2WN3dwFg/build/src'
/usr/bin/gcc -DHAVE_CONFIG_H -I. -I/tmp/tmp.2E2WN3dwFg/src -I..     -g -O2 -MT test-test.o -MD -MP -MF .deps/test-test.Tpo -c -o test-test.o `test -f 'test.c' || echo '/tmp/tmp.2E2WN3dwFg/src/'`test.c
Making all in src
make[2]: Entering directory '/tmp/tmp.2E2WN3dwFg/build/src'
/usr/bin/gcc -DHAVE_CONFIG_H -I. -I/tmp/tmp.2E2WN3dwFg/src -I..     -g -O2 -MT test-test.o -MD -MP -MF .deps/test-test.Tpo -c -o test-test.o `test -f 'test.c' || echo '/tmp/tmp.2E2WN3dwFg/src/'`test.c
mv -f .deps/test-test.Tpo .deps/test-test.Po
/usr/bin/gcc  -g -O2   -o test test-test.o
mv -f .deps/test-test.Tpo .deps/test-test.Po
mv: cannot stat '.deps/test-test.Tpo': No such file or directory
make[2]: *** [GNUmakefile:369: test-test.o] Error 1
make[2]: Leaving directory '/tmp/tmp.2E2WN3dwFg/build/src'
make[1]: *** [GNUmakefile:400: all-recursive] Error 1
make[1]: Leaving directory '/tmp/tmp.2E2WN3dwFg/build'
make: *** [GNUmakefile:320: all] Error 2
make: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/tmp/tmp.2E2WN3dwFg/build/src'
make[1]: Entering directory '/tmp/tmp.2E2WN3dwFg/build'
make[1]: Leaving directory '/tmp/tmp.2E2WN3dwFg/build'

Naturally, running the same build using just a single Make job succeeds:

make -j1 all check

What is strange is that running

make -j8 check

always succeeds, too, resulting in the whole project being built.

  1. How comes such a mature build system still produces Makefile's with races?
  2. And how make check is different from make all check, given that all is a pre-requisite of check anyway?
Solution

  • The answer to question #1 is, I suppose, no one runs make with both all and check on the command line at the same time. That's not a usual use model and so if there are problems, people haven't noticed.

    As you point out, it's not useful since make check will build all first anyway, in the correct order.

    I think that automake-generated makefiles always assume and require that one and only one "top-level" target is provided to the initial make invocation.

    The answer to the second question I expect has to do with the fact that these makefiles are using recursive make. That is, the top level make invocation does not have check: all. It has check: check-recursive and it has all: all-recursive, so those rules can run in parallel. Each one of them invokes a sub-make process and while the submake running check-recursive knows that it can't run the check in parallel with the all target it is running, the other submake running all-recursive doesn't know that some other make instance is also building all in that same directory.