SLURM job. NameError: name 'python3' is not defined

Question

For a while now I've been trying to figure out how to run a job on slurm.

Python versions in the user directory:

$ python -V
Python 2.7.5
$ python3 -V
Python 3.6.8

test.py includes:

import pandas as pd
import numpy as np
true = pd.read_csv("testfile.csv")
print('Just Testing. End for now.')

gpu.job that is passed to SLURM:

#!/usr/bin/python3

#SBATCH --job-name=testjob       # Job name
#SBATCH --output=job.%j.out      # Name of output file (%j expands to jobId)
#SBATCH --cpus-per-task=4        # Schedule one core
#SBATCH --gres=gpu               # Schedule a GPU
#SBATCH --time=71:59:59          # Run time (hh:mm:ss) - run for one hour max
#SBATCH --partition=red          # Run on either the Red or Brown queue
#SBATCH --mail-type=END          # Send an email when the job finishes
#SBATCH --export=ALL             # All of the users environment will be loaded from callers environment


python3 /home/username/test/test.py

After running sbatch gpu.job , I get:

Traceback (most recent call last):

File "/var/spool/slurm/d/job402350/slurm_script", line 13, in

python3 /home/username/test/test.py

NameError: name 'python3' is not defined ~

These variations also did not help and gives the same error:

python3 test.py
/usr/bin/python3 test.py
/usr/bin/python3 /home/username/test/test.py

An advice would be appreciated.

Answer 1

Your submission script is a shell script, not a Python script. So the first line of your submission script should be

#!/usr/bin/env bash

rather than

#!/usr/bin/python3

Technically, you can submit a job script that is a Python script, but then the #SBATCH directives got directly in the Python script and that is the script you submit:

#!/usr/bin/python3

#SBATCH --job-name=testjob       # Job name
#SBATCH --output=job.%j.out      # Name of output file (%j expands to jobId)
#SBATCH --cpus-per-task=4        # Schedule one core
#SBATCH --gres=gpu               # Schedule a GPU
#SBATCH --time=71:59:59          # Run time (hh:mm:ss) - run for one hour max
#SBATCH --partition=red          # Run on either the Red or Brown queue
#SBATCH --mail-type=END          # Send an email when the job finishes
#SBATCH --export=ALL             # All of the users environment will be loaded from callers environment


import pandas as pd
import numpy as np
true = pd.read_csv("testfile.csv")
print('Just Testing. End for now.')

Then you can sbatch this Python script directly. But most often, using a Bash script is preferred to be able to setup the environment, change directories, copy files back and forth, etc. which is easier in Bash than in Python.