I am hoping to clear up some confusion I have about the snakemake resources keyword (documented here). If I am running jobs on a cluster, does this parameter make an actual memory request from the cluster scheduler for a specific job? Or, does it only make sure snakemake won't submit jobs if they would exceed the resource limits specified from the commandline.
I am primarily confused because in my snakemake job logs, it says I only have 100mb memory allocated as resources. But, in my cluster submission command (eg: qsub) I am requesting a couple gb of memory and the jobs do not run out of memory (which I predict they should if there was only 100mb of allocated memory).
Your link is broken, but I'll answer with "it depends". Specifically on your snakemake version and how you are integrating with the cluster. If you use the --cluster
option, you have access to your resource object in the script. I use slurm so my argument looks like sbatch --mem={resources.mem_mb}M ...
. I've not made it to version 8 yet, so I'll defer to someone else on that.
The amount reported in the logs is probably the default, and if you hard code in your memory to qsub it won't matter.