Search code examples
pythonsnakemake

How can I control the frequency about how snakemake submitting the job?

Now, I'm using snakemake cluster to execute with my own k8s.

When submitting the task, the command looks like:

export TMP=/k8s && snakemake --cluster "lsub" --cluster-config cluster-config.yaml -j 2

The brief lsub script logic:

  1. read the parameters from snakemake
  2. submit a k8s job via: cat <<EOF | kubectl create -f - xxxxxxxx EOF (the origin sub-step shell from snakemake is included in command section in xxxxxxxxx)

Due to the k8s environment's throttling policy, I cannot submit too many jobs concurrently. However, it seems that snakemake cannot control it's submitting behavior.(I can control the first step. However, I cannot control the follow-up step)

My idea is changing the lsub script to control the frequency. However, I have no idea about it since I don't know how snakemake read the job status.

So, any suggestion about this situation?

Solution

  • I think you are looking for the --max-jobs-per-second option. If you set that to 0.1 snakemake will submit a job every 10 seconds.

    If instead you mean the number of jobs snakemake will actively submit, you want to modify the --jobs option. --jobs 100 will make sure only 100 jobs are running (submitted) at once.