I am new to SBATCH and bash. I have a simple script to submit the job and I am trying to get the parent directory of the current working directory. It seems from the documentation that '$(dirname $PWD)' is the way to do it. But it always returns null or is empty. I have tried very different versions of it such as '$(dirname '$PWD')' or just $(dirname $PWD). They all result in null. I am suspecting missing something very basic. Any recommendations on what I should check to resolve or debug this better will be helpful. Here is the sample code (simplified)
#!/bin/bash
sbatch <<EOT
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --cpus-per-task=$2
#SBATCH --partition=express
#SBATCH --time=01:00:00
#SBATCH --mem-per-cpu=4G
#SBATCH --output=$1_%A_%a.out
#SBATCH --array=0-4%5
#SBATCH --error=%A_$1_%a.err
#SBATCH --constraint=broadwell
PARENT_DIR="$(dirname $PWD)"
echo "Main directory for submit-test-express script is: $PARENT_DIR"
echo "Current drectory is: $PWD"
echo "Job array ID: $SLURM_ARRAY_JOB_ID , sub-job $SLURM_ARRAY_TASK_ID is running!"
python /path_to_code/code.py $1 $2 $3 $SLURM_ARRAY_TASK_ID /output_folder/ > output_file-$SLURM_ARRAY_JOB_ID-$SLURM_ARRAY_TASK_ID.out
EOT
The code is wrapped in EOT so that the arguments can be passed from other scripts. Also note that the second echo statement prints the current working directory correctly. But the first echo statement returns blank.
Since you are using $PARENT_DIR unscaped your variable gets evaluated before passing the full heredoc to sbatch, try to escape $ like: \$PARENT_DIR