Use wildcard to capture different datasets in Snakemake

Question

I would like to use wildcards in Snakemake in a very simple way to start a script for two datasets. Unfortunately, I cannot find the proper way of doing it.

My data folder contains three files: gene_list.txt, expression_JGI.txt, expression_UBC.txt.

Here is what my snakefile looks like:

rule extract:
    input:
        genes="data/gene_list.txt",
        expression="data/expression_{dataset}.txt"
    output:
        "data/expression_{dataset}_subset.txt"
    shell:
        "bash scripts/extract.sh {input.genes} {input.expression} {output}"

When I use snakemake -c1 extract I get the following error message:

Building DAG of jobs... WorkflowError: Target rules may not contain wildcards. Please specify concrete files or a rule without wildcards at the command line, or have a rule without wildcards at the very top of your workflow (e.g. the typical "rule all" which just collects all results you want to generate in the end).

I tried adding a rule all at the beginning of the snakefile with the desired result files as input without success:

rule all:
    input:
        "data/expression_JGI_subset.txt",
        "data/expression_UBC_subset.txt"

I also tried with expand:

DATASETS = ["JGI", "UBC"]

rule all:
    input:
        expand("data/expression_{dataset}_subset.txt", dataset=DATASETS)

But I get the same error message.

The script works fine when I use it outside Snakemake.

How can I achieve what I want?

Answer 1

When you do snakemake -c1 extract you ask snakemake to execute only rule extract and its dependencies, if any. However, because extract contains wildcards snakemake doesn't know what to replace them with. (Note that rule all is not a dependency of extract).

So either execute snakemake -c1 to run the whole pipeline or specify the concrete files you want to generate, e.g.:

snakemake -c1 -- data/expression_JGI_subset.txt data/expression_UBC_subset.txt