Makefile using m4 to build Dockerfiles is not rebuilding

I have 6 different Dockerfiles, that look very similar, so I used m4 and include to build them from snippets.

I don't know much about using Makefile's for purposes other than C++, and I have a problem with make not rebuilding when a m4 file changed, and not running iterative copy operations unless the target changed.

Please provide some input on better practices and help?

See: https://github.com/ptr727/NxWitness/tree/master/Make

.DEFAULT_GOAL := check

# Even if the m4 files changed make still reports up to date?
INPUTS = $(wildcard ${CURDIR}/*.m4)
OUTPUTS = $(addsuffix .dockerfile, $(basename $(INPUTS)))
# Why does the output look funky?
TARGETS = $(abspath ${CURDIR}/..)/$(basename $(notdir $(INPUTS)))/Dockerfile

%.dockerfile : %.m4
    m4 $< > $@

check:
    @echo "INPUTS = $(INPUTS)"
    @echo "OUTPUTS = $(OUTPUTS)"
    @echo "TARGETS = $(TARGETS)"

create: $(OUTPUTS)

clean:
    -rm *.dockerfile $(OUTPUTS)

# Why does make report up to date even after clean?
# $(foreach file, $(OUTPUTS), $(cp $(file) $(abspath ${CURDIR}/..)/$(basename $(notdir $(file)))/Dockerfile))
replace:
    cp DWSpectrum.dockerfile ../DWSpectrum/Dockerfile
    cp DWSpectrum-LSIO.dockerfile ../DWSpectrum-LSIO/Dockerfile
    cp NxWitness.dockerfile ../NxWitness/Dockerfile
    cp NxWitness-LSIO.dockerfile ../NxWitness-LSIO/Dockerfile
    cp NxMeta.dockerfile ../NxMeta/Dockerfile
    cp NxMeta-LSIO.dockerfile ../NxMeta-LSIO/Dockerfile

build:
    docker build -f ../DWSpectrum/Dockerfile ../DWSpectrum
    docker build -f ../DWSpectrum-LSIO/Dockerfile ../DWSpectrum-LSIO
    docker build -f ../NxWitness/Dockerfile ../NxWitness
    docker build -f ../NxWitness-LSIO/Dockerfile ../NxWitness-LSIO
    docker build -f ../NxMeta/Dockerfile ../NxMeta
    docker build -f ../NxMeta-LSIO/Dockerfile ../NxMeta-LSIO

Solution

In your Dockerfile, you have Make targets like build and replace that don't correspond to real files. Those will always be considered out-of-date and need to be rebuilt, even if their inputs haven't changed. (In GNU Make, marking them as .PHONY: is good practice.)

If you have some task, like building a Docker image, that doesn't directly generate a file on its own, a useful technique is to put some sort of "marker" file to record that it's done. After you do the task, touch the marker file. If one of its inputs has changed, it will appear newer than the marker file, and will cause Docker to rebuild the file.

%/.docker-build: %/Dockerfile
        docker build $(dir $@)
        touch "$@"

You can use similar pattern syntax to generate the Dockerfiles from the m4 files. (This is almost exactly the rule you have above, but with a different target.)

%/Dockerfile: %.m4
        m4 $< >$@

If you have a list of directories

DIRS := DWSpectrum DWSpectrum-LSIO NxWitness NxWitness-LSIO NxMeta NxMeta-LSIO

then you can have the "build" target just depend on the marker files

.PHONY: build
build: $(DIRS:%=%/.docker-build)

and the "clean" rule should clean up everything we generated

.PHONY: clean
clean:
        rm -f $(DIRS:%=%/.docker-build)
        rm -f $(DIRS:%=%/Dockerfile)

I've used two syntaxes here that replace your function calls and foreach loops. When building a rule, if % is on both sides of a rule, it can be replaced by an identical string to make a pattern rule; Make knows that DWSpectrum/.docker-build can be built from DWSpectrum/Dockerfile via this rule. $(DIRS:%=%/Dockerfile) is a substitution reference; for each item in $(DIRS) that matches the pattern % (that is, every item) replace it with the matched string plus the /Dockerfile suffix.