Search code examples
bashsungridengine

Trying to figure out bash scripting with Sun Grid Engine

I'm trying to use integer variables within a bash script for submitting to Sun Grid Engine (using qsub). Can someone clarify why this does not work?

NUMCORES=32

#$ -pe mpi $NUMCORES

(gives the error "Unable to read script file because of error: Numerical value invalid! The initial portion of string "$NUMCORES" contains no decimal number)

But this does:

#$ -pe mpi 32

I've looked at some solutions which involve awk or bc but can't seem to make them work. Obviously I am new to bash scripting - thanks for any help!

Solution

  • The character # in a bash script signals comments. Hence, in the line #$ -pe mpi $NUMCORES, the value of the environement variable NUMCORES will not be used, since it is a comment.

    Yet, these comments are useful to pass parameters to the qsub command. See the man page and these examples for instance. Notice that these parameters can be passed on the command line:

    qsub -pe mpi 32 myjob.sh

    The alternative is to use a command before qsub to generate the job. Let's imagine that the job script jobgeneric.sh is:

    #!/bin/bash
#$ -N useless
#$ -cwd
#$ -pe mpi $NUMCORES
#$ -l h_vmem=6G

echo 42

    Let's use the sed command to substitute the environment variable and create a new file job.sh:

    export NUMCORES=32
sed 's/$NUMCORES/'$NUMCORES'/g' jobgeneric.sh > job.sh
cat job.sh

    See also Environment variable substitution in sed . The result is:

    #!/bin/bash
#$ -N useless
#$ -cwd
#$ -pe mpi 32
#$ -l h_vmem=6G

echo 42

    Now the job can be submitted by using qsub job.sh!