We have a Makefile where dependencies are marked correcty and it would run nicely on any number of CPU cores. We currently start the Makefile with
CPUS ?= $(shell nproc)
MAKEFLAGS += -j $(CPUS) -l $(CPUS) -s
and this used to work nicely in GNU Make 4.1 or lesser. However, GNU Make 4.2 or newer will emit following warning:
warning: -j12 forced in makefile: resetting jobserver mode.
(the number in the warning obviously depends on CPU cores in your system). The processing seems to result in correct output but I don't want console spam. Fortunately, this only happens for recursive use where a target contains rules that read $(MAKE) ... but this still happens often enough to spam the console.
Is it possible to default to nproc cores (or maybe nproc + 1 might be optimal to overlap some IO with CPU load?) if top level make is executed without the -j and -l flags without triggering any warnings?
The Makefile supports a lot of targets and I would want to default all the targets to parallel processing scaled according to the current system.
I was able create this hack:
ifndef MAKEFLAGS
CPUS ?= $(shell nproc)
MAKEFLAGS += -j $(CPUS) -l $(CPUS) -s
$(info Note: running on $(CPUS) CPU cores by default, use flag -j to override.)
endif
It seems to work correctly in sense if I run make alone, I get parallel execution on multiple CPUs and if I run make -j2 I get execution on two cores. I don't know if this would cause additional side-effects if some other make flags have been used but in worst case it should default to basic GNU Make behavior of running on single core only and lose the -s flag in this example.
Note that according to the documentation one should be able to write
ifeq (,$(findstring j,$(MAKEFLAGS)))
CPUS ?= $(shell nproc)
MAKEFLAGS += -j $(CPUS)
endif
to only adjust CPU count if not otherwise defined but in reality this doesn't work because for some yet unknown reason $(MAKEFLAGS) doesn't include the -j flag outside the recipes (I tested with GNU Make 4.1)!
Try Makefile like this:
$(warning MAKEFLAGS=$(MAKEFLAGS))
$(warning $(shell printf "[%s] " $(MAKEFLAGS)))
default:
@printf "MAKEFLAGS in recipe: "
@printf "[%s] " $(MAKEFLAGS)
@echo
and the output of make -j4 will be
Makefile:1: MAKEFLAGS=
Makefile:2: []
MAKEFLAGS in recipe: [-j] [--jobserver-fds=3,4]
Note how builtin functions such as findstring or shell printf cannot access the -j flag. As a result, you have to assume that if MAKEFLAGS has been set, the end user probably knows how he wants to set the flags and probably has set the -j correctly.