Search code examples
slurmhpc

SLURM dependency on nonexistent job ID


Submitting a job with sbatch --dependency=afterok:$dependencylist should make the current submission wait until jobs with IDs in $dependencylist have concluded with an exit code 0. What happens if some or all IDs in $dependencylist are not valid job IDs or have never been submitted? Will SLURM recognize it and start the jobs?

Alternatively, how can I provide empty arguments to sbatch? I tried sbatch --dependency=afterok:"", but SLURM complains that sbatch: error: Batch job submission failed: Job dependency problem. The reason I want to keep the --dependency option is that I'm writing a more general script where at times I will have some dependencies and other times I won't.


Solution

  • What happens if some or all IDs in $dependencylist are not valid job IDs or have never been submitted?

    From a test made with Slurm 20.02.7, it appears that is the job ID is unknown from slurmctld (either not submitted yet, or job from earlier in the past than what is configured as MinJobAge), the option is silently ignored. scontrol show job says then Dependency=(null). This does not change afterwards, even if a job with that ID appears.

    Alternatively, how can I provide empty arguments to sbatch?

    You can take advantage of the above-described behaviour by using "0" as job ID when no dependency is required. Depending on the version, you can also try -d afterok:-1, or, try setting a dependency on a past job that successfully completed.