Search code examples
snakemake

snakemake use both --keep-going and --stats

The --keep-going flag tells snakemake to go on with independent jobs if a job fails.
The --stats /path_to_the_runtime_statistics_file option produces the runtime statistics of all the rules at the end of the pipeline.

However, if a job fails then the pipeline does not produce the runtime statistics file at all.
I.e. if you have 100 jobs and only one of them fails, then the runtime statistics about the 99 successful jobs are not produced.
How should one get the runtime statistics of the jobs succeeded?
Thanks in advance.

Solution

  • if you see the snake make API documentation of how --stats is called execute function implementation. You will get to know that, the implementation calls snakemake.stats module implemtation over a condition which says if success:! So, the straight answer to your question is NO you can't do it!

    Two way of moving forward:

    1. Quick & simple solution uses their stats implementation, and write what you wanted to do taking/calling particular functionality as per your needs! :)
    from snakemake import stats
and do whatever you want .....
    1. If you can't, then create an issue on snakemake github, Based on the priority their developer will add this feature to the newer versions of snakemake! It very slow process.