Search code examples
bashpbsqsubtorque

Setting PBS/Torque/qsub parameters in script via command line arguments


I want to be able to easily change how many nodes, ppn, etc I submit to qsub via script. That is, I want run somthing like this:

qsub script.sh --name=test_job --nodes=2 --ppn=2 --arg1=2

With a script like the following:

#/bin/bash
#PBS -N ${NAME}
#PBS -l nodes=${NODES}:ppn=${PPN},walltime=${WALLTIME}
#PBS -q ${QUEUE}
#PBS -m ${MAILOPTS}
#PBS -M ${EMAIL}

/some/command ${ARG1}

So, I want to be able to pass in arguments that both change the PBS environment as well as some that go to the executable itself.

I've tried using the -v argument of qsub:

qsub script.sh -v NAME=test_job,NODES=16,PPN=16,ARG1=2

But the job submitted with the name script.sh and 1 node, 1 ppn.

Any ideas on a solution to this?


Solution

  • @dbeer's answer gave me a little more insight. The solution to my problem is as follows:

    #PBS args are overwritten by the command line. In such case, the args to PBS and the script itself must be separated. Therefore, rather than trying to do something like:

    #PBS -l nodes=${NODES}:ppn=${PPN},walltime=${WALLTIME}
    
    /some/command ${ARG1}
    

    inside the script and running like

    qsub script.sh -v NODES=2,PPN=2,WALLTIME=160:00:00,ARG1=2
    

    these can all be set with args to qsub itself:

    qsub script.sh -l nodes=2:ppn=2,walltime=160:00:00
    

    Then, any args that need to be passed to the executable can be passed through the -v argument to qsub:

    qsub script.sh -l nodes=2:ppn=2,walltime=160:00:00 -v ARGS1=2