Running external scripts with wildcards in snakemake

Question

I am trying to run a snakemake rule with an external script that contains a wildcard as noted in the snakemake reathedocs. However I am running into KeyError when running snakemake.

For example, if we have the following rule:

SAMPLE = ['test']

rule all:
    input:
        expand("output/{sample}.txt", sample=SAMPLE)

rule NAME:
    input: "workflow/scripts/{sample}.R"
    output: "output/{sample}.txt",
    script: "workflow/scripts/{wildcards.sample}.R"

with the script workflow/scripts/test.R containing the following code

out.path = snakemake@output[[1]]

out = "Hello World"

writeLines(out, out.path)

I get the following error when trying to execute snakemake.

Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job counts:
    count   jobs
    1   NAME
    1   all
    2

[Fri May 21 12:04:55 2021]
rule NAME:
    input: workflow/scripts/test.R
    output: output/test.txt
    jobid: 1
    wildcards: sample=test

[Fri May 21 12:04:55 2021]
Error in rule NAME:
    jobid: 1
    output: output/test.txt

RuleException:
KeyError in line 14 of /sc/arion/projects/LOAD/Projects/sandbox/Snakefile:
'wildcards'
  File "/sc/arion/work/andres12/conda/envs/py38/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 2231, in run_wrapper
  File "/sc/arion/projects/LOAD/Projects/sandbox/Snakefile", line 14, in __rule_NAME
  File "/sc/arion/work/andres12/conda/envs/py38/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 560, in _callback
  File "/sc/arion/work/andres12/conda/envs/py38/lib/python3.8/concurrent/futures/thread.py", line 57, in run
  File "/sc/arion/work/andres12/conda/envs/py38/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 546, in cached_or_run
  File "/sc/arion/work/andres12/conda/envs/py38/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 2262, in run_wrapper
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /sc/arion/projects/LOAD/Projects/sandbox/.snakemake/log/2021-05-21T120454.713963.snakemake.log

Does anyone know why this not working correctly?

Answer 1

I agree with Dmitry Kuzminov that having a script depending on a wildcard is odd. Maybe there are better solutions.

Anyway, this below works for me on snakemake 6.0.0. Note that in your R script snakemake@output[1] should be snakemake@output[[1]], but that doesn't give the problem you report.

SAMPLE = ['test']

rule all:
    input:
        expand("output/{sample}.txt", sample=SAMPLE)

rule make_script:
    output:
        "workflow/scripts/{sample}.R",
    shell:
        r"""
echo 'out.path = snakemake@output[[1]]' > {output}
echo 'out = "Hello World"' >> {output}
echo 'writeLines(out, out.path)' >> {output}
        """

rule NAME:
    input: 
        "workflow/scripts/{sample}.R"
    output: 
        "output/{sample}.txt",
    script: 
        "workflow/scripts/{wildcards.sample}.R"