Search code examples
pythonpipelinesnakemake

Snakemake: how to create rule without explicit output file, and only with specified input, and log files?


I would like to create a Snakemake rule where there are: input, log, shell sections. There is no output, I would like to catch the log only as a result of the command.


Solution

  • You could skip output: and just use log: in the rule. These log files can be used as targets or as input to other rules. As per the doc:

    Log files can be used as input for other rules, just like any other output file. However, unlike output files, log files are not deleted upon error. This is obviously necessary in order to discover causes of errors which might become visible in the log file.

    So the code would look like:

    rule some_rule:
        input: "a.txt"
        log: "a.log"
        shell: "mycommand {input} > {log}"
    

    Advantage here is that, unlike output file, log file will be preserved in case of job failure. However this advantage is also a disadvantage, because if you rerun the pipeline, snakemake will not rerun the failed job, as the rule's output file (ie. log file here) is already present. So, unless log preservation is important when a job fails, you might be better served with solution suggested by Maarten-vd-Sande.