I was wondering if it was possible to run multiple snakemake pipelines one after another in a row. I have 3 snakemake pipelines, I would like the 1st one to finish, then the 2nd one starts in automatic, then the 3rd one will start as soon as the 2nd one finishes in conda environment. The inputs and outputs might not necessarily be the same.
Pseudocode something like:
snakemake -s Snakefile_pipeline_1 -j 15
snakemake -s Snakefile_pipeline_2 -j 40
conda activate some_env
snakemake -s Snakefile_pipeline_3 -j 10
If this is meant to execute sequentially and condition on successful execution of the previous step, then it can be saved as a basic shell script and executed as such:
snakemake -s Snakefile_pipeline_1 -j 15 &&
snakemake -s Snakefile_pipeline_2 -j 40 &&
conda activate some_env &&
snakemake -s Snakefile_pipeline_3 -j 10
If it's possible that these rules could be executed simultaneously (so they share some common output), then it's best to create another Snakefile that imports the other Snakefiles as modules:
module other_workflow:
snakefile:
# here, plain paths, URLs and the special markers for code hosting providers (see below) are possible.
"other_workflow/Snakefile"
use rule * from other_workflow exclude ruleC as other_*