I am trying to set up a multi-stage data processing pipeline using generic rules in GNU make. However, I get the error:
Makefile:29: *** multiple target patterns. Stop.
I suspect this is due to errors I made in the nesting of the $(foreach...) and $(call...) functions.
.
├── Makefile
├── data
│ ├── processed
│ └── raw
│ ├── annotations.csv
│ └── observations.csv
└── src
├── init.py
└── parser.py
# $1 targets
# $2 lauch_code
# $3 code_prereq
# $4 data_prereq
# $5 arguments
# Blank line at the beginning to have line breaks in $(foreach ...)!
define conprod
$1: $2 $3 $4
mkdir -p $(dir $1)
python3 $2 \
$(foreach dp, $4, --consume $(dp)) \
$(foreach t, $1, --produce $(t)) \
$5
endef
params = 60 600
define target_pattern
data/processed/resam_$(param)s/annotated_observations.csv
endef
launch_code = src/parser.py
code_prereq =
data_prereq = data/raw/observations.csv data/raw/annotations.csv
$(info Info start)
$(info $(foreach param,$(params),$(call conprod,$(call target_pattern,$(param)),$(launch_code),$(code_prereq),$(data_prereq),$(param))))
$(info Info end)
$(foreach param,$(params),$(call conprod,$(call target_pattern,$(param)),$(launch_code),$(code_prereq),$(data_prereq),$(param)))
.PHONY: data
data: $(foreach param,$(params),$(call target_pattern,$(param)))
/mnt/c/Users/Public/dummy_projects/minimake$ make data -Bnd
GNU Make 4.2.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Reading makefile 'Makefile'...
Info start
data/processed/resam_60s/annotated_observations.csv: src/parser.py data/raw/observations.csv data/raw/annotations.csv
mkdir -p data/processed/resam_60s/
python3 src/parser.py --consume data/raw/observations.csv --consume data/raw/annotations.csv --produce data/processed/resam_60s/annotated_observations.csv 60
data/processed/resam_600s/annotated_observations.csv: src/parser.py data/raw/observations.csv data/raw/annotations.csv
mkdir -p data/processed/resam_600s/
python3 src/parser.py --consume data/raw/observations.csv --consume data/raw/annotations.csv --produce data/processed/resam_600s/annotated_observations.csv 600
Info end
Makefile:29: *** multiple target patterns. Stop.
I'm going to go out on a limb and guess that you are very new to Make.
If I am able to guess what you actually want, it would look simply like
params := 60 600
.PHONY: all data
all: data
data: $(patsubst %,data/processed/resam_%/annotated_observations.csv,$(params))
data/processed/resam_%/annotated_observations.csv: src/parser.py data/raw/observations.csv data/raw/annotations.csv
mkdir -p data/processed/resam_$*
python3 $< $(patsubst %,--consume %,$(filter-out $<,$^)) --produce $@ $*
(Stack Overflow renders tabs as spaces, so you won't be able to copy/paste this directly into your Makefile without modifications.)
This is quite clumsy still; you would probably be able to refactor this into something significantly simpler and more elegant by avoiding the pesky deep subdirectory structures.