How to load anaconda virtual environment from slurm?

Question

I set up anaconda on the VM where slurm is installed as follows.

base                     /root/anaconda3
upload                *  /root/anaconda3/envs/upload

And the file in slurm is:

#SBATCH -J vs_slurm_upload
#SBATCH -o ./out/%j_log.out
#SBATCH --ntasks=1
#SBATCH --array=0-14
FILES=(../workdir/*)

pwd
conda info --envs
source activate upload

However, unlike the Anaconda settings I set, there is no upload virtual environment.

Here is the result:

base                     /root/anaconda3

In order to set the virtual environment in slurm, do I need to set docker instead of setting it in my VM?

I don't quite understand slurm yet.

Thank you for your reply.

Answer 1

The difference is perhaps because the user-specific ~/.condarc is not being loaded due to not running the SLURM script in login mode (i.e., as your user). Try modifying the script to something like:

#!/bin/bash -l

#SBATCH -J vs_slurm_upload
#SBATCH -o ./out/%j_log.out
#SBATCH --ntasks=1
#SBATCH --array=0-14
FILES=(../workdir/*)

pwd
conda info --envs
conda activate upload

Note that I also changed the source activate to conda activate - the former syntax has been deprecated since Conda v4.4.