Search code examples
rterminaljobslsf

submit job arrays in LSF

I have 12 folders that all contains one R file with the same name (e.g., file.R). The name of the folders are m1, m2, ..., m12. To run each file, I run bsub -n 2 -q long -W 12:00 -R "rusage[mem=25000]" -M 25000 -hl R CMD BATCH file.R in each folder. Is there a way to run it as a job array in the LSF submission system using bsub? Thank you.

Solution

  • Is there a way to run it as a job array in the LSF submission system using bsub?

    Yes there is. Use -J "[1-12]" to run an array job. The job will run 12 instances. You'll need to write a simple script to set the CWD properly and then start R. Something like this should work. The instance number is available through the environment variable $LSB_JOBINDEX.

    $ cat runjob.sh 
#!/bin/sh

cd m${LSB_JOBINDEX}
exec R CMD BATCH file.R

    and then submit your job like this

    $ bsub -n 2 -q long -W 12:00 -R "rusage[mem=25000]" -M 25000 -hl -J "[1-12]" sh runjob.sh 
Job <1164> is submitted to queue <long>.