GNU make: Generating automatic dependencies with generated header files

Question

So I followed the Advanced Auto-Dependency Generation paper --

Makefile:

SRCS := main.c foo.c

main: main.o foo.o

%.o: %.c
    $(CC) -MMD -MG -MT '$@ $*.d' -c $< -o $@
    cp $*.d $*.tmp
    sed -e 's;#.*;;' -e 's;^[^:]*: *;;' -e 's; *\\$$;;' \
        -e '/^$$/d' -e 's;$$; :;' < $*.tmp >> $*.d
    rm $*.tmp

clean::
    -rm *.o *.d main

-include $(SRCS:.c=.d)

main.c:

#include "foo.h"

int main(int argc, char** argv) {
  foo() ;
  return 0 ;
}

foo.h:

#ifndef __FOO_H__
#define __FOO_H__

void foo() ;

#endif

-- and it works like a charm.

---

But when foo.h becomes a generated file --

Makefile:

...

HDRS := foo.h

$(HDRS):
    mk_header.sh $*

clean::
    -rm $(HDRS)
...

mk_header.sh:

#!/bin/bash
UP=$(tr "[:lower:]" "[:upper:]" <<< $1)

cat <<EOF > $1.h
#ifndef __${UP}_H__
#define __${UP}_H__

void $1() ;

#endif
EOF

The 1st time I run make, main.d is not yet generated, and thus foo.h is not considered a prerequisite, and thus isn't been generated:

$ ls
foo.c  main.c  Makefile  mk_header.sh*

$ make
cc -MMD -MG -MT 'main.o main.d' -c main.c -o main.o
cp main.d main.tmp
sed -e 's;#.*;;' -e 's;^[^:]*: *;;' -e 's; *\\$;;' \
    -e '/^$/d' -e 's;$; :;' < main.tmp >> main.d
rm main.tmp
cc -MMD -MG -MT 'foo.o foo.d' -c foo.c -o foo.o
cp foo.d foo.tmp
sed -e 's;#.*;;' -e 's;^[^:]*: *;;' -e 's; *\\$;;' \
    -e '/^$/d' -e 's;$; :;' < foo.tmp >> foo.d
rm foo.tmp
cc   main.o foo.o   -o main

$ ls
foo.c  foo.d  foo.o  
main*  main.c  main.d  main.o  
Makefile  mk_header.sh*

Only in the 2nd invocation of make, the foo.h is generated, and as a result another build cascades.

$ make
./mk_header.sh foo
cc -MMD -MG -MT 'main.o main.d' -c main.c -o main.o
cp main.d main.tmp
sed -e 's;#.*;;' -e 's;^[^:]*: *;;' -e 's; *\\$;;' \
    -e '/^$/d' -e 's;$; :;' < main.tmp >> main.d
rm main.tmp
cc   main.o foo.o   -o main

$ ls
foo.c  foo.d  foo.h  foo.o  
main*  main.c  main.d  main.o  
Makefile  mk_header.sh*

And only after that make realizes that:

$ make
make: `main' is up to date.

---

So my question is: Is there a way to extend the recipe suggested by the paper above, to allow for generated header files, without the elimination of the performance gain realized by not having to re-evaluate the entire make tree when including the *.d fragments?

Answer 1

The problem is that the *.d Makefile-fragments generation must be performed after all the header generation is complete. Putting it this way, one can use the make dependencies to force the right order:

SRCS := main.c foo.c
HDRS := foo.h

main: main.o foo.o

%.o: %.c | generated_headers
    $(CC) -MMD -MG -MT '$@ $*.d' -c $< -o $@
    cp $*.d $*.tmp
    sed -e 's;#.*;;' -e 's;^[^:]*: *;;' -e 's; *\\$$;;' \
        -e '/^$$/d' -e 's;$$; :;' < $*.tmp >> $*.d
    rm $*.tmp

-include $(SRCS:.c=.d)

$(HDRS):
    mk_header.sh $*

generated_headers: $(HDRS)

clean:
    -rm $(HDRS) *.o *.d main

.PHONY: clean generated_headers

Notes:

1. I use an order-only dependency.

2. This solution is fairly scalable: Each generate-header rule, needs only to be a prerequisite of the generated_headers .PHONY target. Assuming that the header generation rule is written properly, once it has been generated correctly, satisfying the generated_headers target should be a no-op.

3. One can't compile a single object, even if that object does not require any generated headers, without generating all the generated headers of the project first. While this is technically sound, your developers will complain.

   So you should think about having a FAST_AND_LOOSE flag, that will turn this feature off:

   %.o: %.c | $(if $(FAST_AND_LOOSE),,generated_headers)
       ...
   

   Thus a developer may issue:

   make FAST_AND_LOOSE=1 main.o