How does Shake decide whether to rebuild a target?

Question

I'm confused about the rules Shake uses to work out whether an output needs to be rebuilt. I have a simple build for documents with two steps. The full build file is below but to summarise, asciidoc is used to transform a .txt file into a .dbxml (Docbook XML) file, which is then transformed into PDF using dblatex.

I expect that if I touch the PDF and rerun shake, nothing should happen because the output is newer than both inputs. However, shake in fact executes the dblatex step.

Next, I expect that if I touch the .dbxml file then shake will execute dblatex but not asciidoc, because the dbxml is newer than its input (i.e. the .txt). However shake in fact executes both asciidoc and dblatex steps.

Have I made a mistake in my dependencies?

import Development.Shake
import Development.Shake.FilePath

-- List of output files
outputs = ["process.pdf"]

main = shakeArgs shakeOptions{shakeVerbosity=Diagnostic} $ do
    want outputs

    -- Rule to produce pdf files from dbxml inputs
    "*.pdf" *> \out -> do
        let dbxml = out `replaceExtension` "dbxml"
        need [dbxml]
        cmd "dblatex" "-o" out dbxml

    -- Rule to produce dbxml files from txt (asciidoc) inputs
    "*.dbxml" *> \out -> do
        let src = out `replaceExtension` "txt"
        need [src]
        cmd "asciidoc" "--backend=docbook45" "--doctype=article" "-o" out src

Answer 1

In Shake a file is considered dirty if its last modified time changes from when it was built. In make a file is considered dirty if its last modified time is older than its dependencies. I suspect your observations all stem from this difference. To directly answer the question, Shake rebuilds a file if it or any of its direct dependencies have changed.

Why does Shake do something different to make? Three reasons:

1. Make doesn't store enough information to allow it to rebuild if the modification time changes since it doesn't record any extra metadata about what the modification time was in the last build.
2. By doing time comparison make relies on a monotonically increasing clock, which breaks if the user changes their system time, and seems particularly prone to breakage on NFS filesystems.
3. Assume A depends on B, and B depends on C. C changes, and B rebuilds, but the rule for B is clever enough to spot that the old B doesn't need updating. In make you have two bad options - touch B anyway and rebuild A, or don't touch B and then run the rule for B every time you run make. In Shake you don't touch B, A doesn't rebuild, and B doesn't rebuild next time.

As for your build system, it all looks good to me. My only minor tweak would be the use of the infix operator -<.> instead of replaceExtension - they are both the same function but the operator looks clearer to my eye.