Search code examples
makefilepandoc

makefile with subdirectories

I am working on a book. The chapters will be written in Markdown (.md), and then converted to both html and pdf (via LaTeX) versions using pandoc. Each chapter has a handful of associated Python scripts that generate some images and need to be run before the chapter is built. I am trying to write a makefile that will compile all the chapters to these two formats.

For now, the project is structured as follows:

project
  |--- makefile
  |--- chapters
    | --- chapter1
      | --- main.md
      | --- genimage.py
      | --- genanotherimage.py
    | --- chapter2
      |--- main.md
      |--- otherimage.py
    | --- output
      | --- html
        | --- chapter1.html
        | --- chapter2.html
      | --- pdf
        | --- chapter1.pdf
        | --- chapter2.pdf

I would like to type "make chapter1" (or similar) and have it refresh both output/html/chapter1.html and output/pdf/chapter1.pdf, re-running all the .py scripts in the corresponding directory if they have changed. Ideally I would have one rule that handles all the chapters in parallel rather than a separate one for each one. (The actual command generate the html/pdf is "pandoc -o output/html/chapter1.html chapter1/main.md" and so on.)

I am not very familiar with make and my attempts so far have been very unsuccessful. I can't manage to make a target where there are multiple files to update, and I have not managed to use patterns to handle each chapter with a single rule. I am happy to reorganize somewhat if it makes things easier.

Is this workflow possible with a makefile? I am grateful for any hints to get started; I'm at a loss and even just knowing the right things to look up in the manual would be very helpful.

Solution

  • The following is based on the project tree you show and assumes GNU make. It also assumes that you must run pandoc and your python scripts from the top level directory of the project. Pattern rules can probably help:

    CHAPTERS := $(notdir $(wildcard chapters/chapter*))
.PHONY: all $(CHAPTERS)

all: $(CHAPTERS)

$(CHAPTERS): %: chapters/output/html/%.html chapters/output/pdf/%.pdf

chapters/output/html/%.html chapters/output/pdf/%.pdf: chapters/%/main.md
    for python_script in $(wildcard $(<D)/*.py); do ./$$python_script; done
    mkdir -p chapters/output/html chapters/output/pdf
    pandoc -o chapters/output/html/$*.html <other-options> $<
    pandoc -o chapters/output/pdf/$*.pdf <other-options> $<

    The main subtlety is that when GNU make encounters a pattern rule with several targets it considers that one single execution of the recipe builds all targets. In our case the HTML and PDF outputs are produced by the same execution of the recipe.

    Note: with recent versions of GNU make rules with grouped targets (&:) do the same.

    This is not 100% perfect because a chapter will not be rebuilt if you modify or add python scripts. If you also need this we will need more sophisticated GNU make features like secondary expansion or eval.

    Example with secondary expansion:

    CHAPTERS := $(notdir $(wildcard chapters/chapter*))
.PHONY: all $(CHAPTERS)

all: $(CHAPTERS)

$(CHAPTERS): %: chapters/output/html/%.html chapters/output/pdf/%.pdf

.SECONDEXPANSION:

chapters/output/html/%.html chapters/output/pdf/%.pdf: chapters/%/main.md $$(wildcard chapters/$$*/*.py)
    for python_script in $(wildcard $(<D)/*.py); do ./$$python_script; done
    mkdir -p chapters/output/html chapters/output/pdf
    pandoc -o chapters/output/html/$*.html $<
    pandoc -o chapters/output/pdf/$*.pdf $<

    To understand why $$ in $$(wildcard chapters/$$*/*.py) see the GNU make manual.

    Example with eval:

    CHAPTERS := $(notdir $(wildcard chapters/chapter*))
.PHONY: all $(CHAPTERS)

all: $(CHAPTERS)

$(CHAPTERS): %: chapters/output/html/%.html chapters/output/pdf/%.pdf

# $1: chapter
define CHAPTER_RULE
PYTHON_SCRIPTS_$1 := $$(wildcard chapters/$1/*.py)

chapters/output/html/$1.html chapters/output/pdf/$1.pdf: chapters/$1/main.md $$(PYTHON_SCRIPTS_$1)
    for python_script in $$(PYTHON_SCRIPTS_$1); do ./$$$$python_script; done
    mkdir -p chapters/output/html chapters/output/pdf
    pandoc -o chapters/output/html/$1.html $$<
    pandoc -o chapters/output/pdf/$1.pdf $$<
endef
$(foreach c,$(CHAPTERS),$(eval $(call CHAPTER_RULE,$c)))

    To understand why $$ or $$$$ see the GNU make manual.