Search code examples
bashsungridengineqsub

Use variables as argument parameters inside qsub script


I want to pick up a number of models from a folder and use them in an sge script for an array job. So I do the following in the SGE script:

MODELS=/home/sahil/Codes/bistable/models
numModels=(`ls $MODELS|wc -l`)
echo $numModels 

#$ -S /bin/bash
#$ -cwd 
#$ -V
#$ -t 1-$[numModels] # Running array job over all files in the models directory.

model=(`ls $MODELS`)
echo "Starting ${model[$SGE_TASK_ID-1]}..."

But I get the following error:

Unable to read script file because of error: Numerical value invalid!
The initial portion of string "$numModels" contains no decimal number

I have also tried to use

#$ -t 1-${numModels}

and

#$ -t 1-(`$numModels`)

but none of these work. Any suggestions/alternate methods are welcome, but they must use the array job functionality of qsub.


Solution

  • Beware that to Bash, #$ -t 1-$[numModels] is nothing more than a comment; hence it does not apply variable expansion to numModels.

    One option is to pass the -t argument in the command line: remove it from your script:

    #$ -S /bin/bash
    #$ -cwd 
    #$ -V
    
    model=(`ls $MODELS`)
    echo "Starting ${model[$SGE_TASK_ID-1]}..."
    

    and submit the script with

    MODELS=/home/sahil/Codes/bistable/models qsub -t 1-$(ls $MODELS|wc -l) submit.sh
    

    If you prefer to have a self-contained submission script, another option is to pass the content of the whole script through stdin like this:

    #!/bin/bash
    qsub <<EOT
    MODELS=/home/sahil/Codes/bistable/models
    numModels=(`ls $MODELS|wc -l`)
    echo $numModels 
    
    #$ -S /bin/bash
    #$ -cwd 
    #$ -V
    #$ -t 1-$[numModels] # Running array job over all files in the models directory.
    
    model=(`ls $MODELS`)
    echo "Starting ${model[$SGE_TASK_ID-1]}..."
    EOT
    

    Then you source or execute that script directly to submit your job array (./submit.sh rather than qsub submit.sh as the qsub command is here part of the script.