Search code examples
javacondawrappersnakemake

Set java_opts for baserecalibrator snakemake wrapper

I am running a snakemake pipeline with some conda wrappers and I'm trying to allocate some more memory to a rule since it's taking ages. I tried to allocate more memory and threads through the java opts, but the rule fails. I can't provide an error message. I'm running on a cluster and the log says I should look in the log files of the involved rules. This log file is empty though.

This is my code for the rule:

rule recalibrate_base_qualities:
    (unchanged)
    params:
        (unchanged)
        java_opts="-Xmx4G -XX:ParallelGCThreads=8", # also tried just: java_opts="-Xmx4G",
    threads: 8
    resources:
        mem_mb=32768,
        time="12:00:00", # just for cluster scheduling
    wrapper:
        "0.74.0/bio/gatk/baserecalibrator"

Can anyone help me spot the error?

Thanks in advance

Solution

  • The wrapper code imports some utils from here. Specifically, if you set mem_mb in resources, it adds -Xmx{mem_mb}M to your java invocation. If you also specify -Xmx in java_opts, the code will exit with a message about having both mem_mb and -Xmx. Not sure why you don't see that in the log, but that is why your job is failing.

    You can, however, specify -XX:ParallelGCThreads=8 alone and it should work, taking the memory from your resources.

    So, if you specify mem_mb or mem_gb in resources, do not include -Xmx in your java_opts.