My jobs are expensive in terms of internet bandwidth. So I do not want one execution host to run more than two of them parallelly. Is there a way to tell bsub to do so?
#BSUB -J myjob[1-100]%10
#BSUB -n 4
The way bsub works is that it will schedule all 10 jobs to the first execution host if it has 40(10 x 4) job slots available. I rather want to distribute my 10 parallel jobs over five hosts.
So is there a way in bsub to specify maximum jobs per host from the pool of my parallel jobs?
The best answer I found is to use BSUB -R "order[!slots]" to select the hosts with most available slots at execution time
The ! is for sorting at every job within the pool of parallel jobs. If it is omitted: BSUB -R "order[slots]", the sorting will happen only once when the job is submitted.