Search code examples
makefilemetaprogramminggnu-maketemplate-meta-programming

GNU make: nesting of $(foreach...) and $(call...) produces Error: *** multiple target patterns. Stop


Problem

I am trying to set up a multi-stage data processing pipeline using generic rules in GNU make. However, I get the error:

Makefile:29: *** multiple target patterns. Stop.

I suspect this is due to errors I made in the nesting of the $(foreach...) and $(call...) functions.

Minimal example

Project hierarchy

.
├── Makefile
├── data
│   ├── processed
│   └── raw
│       ├── annotations.csv
│       └── observations.csv
└── src
    ├── init.py
    └── parser.py

The Makefile

# $1 targets
# $2 lauch_code
# $3 code_prereq
# $4 data_prereq
# $5 arguments
# Blank line at the beginning to have line breaks in $(foreach ...)!
define conprod

$1: $2 $3 $4
    mkdir -p $(dir $1)
    python3 $2 \
        $(foreach dp, $4, --consume $(dp)) \
        $(foreach t, $1, --produce $(t)) \
        $5
endef


params = 60 600
define target_pattern
data/processed/resam_$(param)s/annotated_observations.csv
endef
launch_code = src/parser.py
code_prereq =
data_prereq = data/raw/observations.csv data/raw/annotations.csv

$(info Info start)
$(info $(foreach param,$(params),$(call conprod,$(call target_pattern,$(param)),$(launch_code),$(code_prereq),$(data_prereq),$(param))))
$(info Info end)

$(foreach param,$(params),$(call conprod,$(call target_pattern,$(param)),$(launch_code),$(code_prereq),$(data_prereq),$(param)))

.PHONY: data
data: $(foreach param,$(params),$(call target_pattern,$(param)))

Console output

/mnt/c/Users/Public/dummy_projects/minimake$ make data -Bnd
GNU Make 4.2.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Reading makefile 'Makefile'...
Info start

data/processed/resam_60s/annotated_observations.csv: src/parser.py  data/raw/observations.csv data/raw/annotations.csv
        mkdir -p data/processed/resam_60s/
        python3 src/parser.py  --consume data/raw/observations.csv  --consume data/raw/annotations.csv  --produce data/processed/resam_60s/annotated_observations.csv 60
data/processed/resam_600s/annotated_observations.csv: src/parser.py  data/raw/observations.csv data/raw/annotations.csv
        mkdir -p data/processed/resam_600s/
        python3 src/parser.py  --consume data/raw/observations.csv  --consume data/raw/annotations.csv  --produce data/processed/resam_600s/annotated_observations.csv 600
Info end
Makefile:29: *** multiple target patterns.  Stop.

Leads

Possible causes I could rule out:

  • The indentations are tabs in the IDE (works fine with other targets)
  • The file paths inside the subsystem don't contain colons (as shown by the console output)

"Invisible" stuff

  • Due to the string produced by $(foreach ...), there is a whitespace at the end of the line (after the argument 60 or 600 respectively)

Solution

  • I'm going to go out on a limb and guess that you are very new to Make.

    If I am able to guess what you actually want, it would look simply like

    params := 60 600
    .PHONY: all data
    all: data
    data: $(patsubst %,data/processed/resam_%/annotated_observations.csv,$(params))
    
    data/processed/resam_%/annotated_observations.csv: src/parser.py data/raw/observations.csv data/raw/annotations.csv
        mkdir -p data/processed/resam_$*
        python3 $< $(patsubst %,--consume %,$(filter-out $<,$^)) --produce $@ $*
    

    (Stack Overflow renders tabs as spaces, so you won't be able to copy/paste this directly into your Makefile without modifications.)

    This is quite clumsy still; you would probably be able to refactor this into something significantly simpler and more elegant by avoiding the pesky deep subdirectory structures.