Search code examples
multithreadingparallel-processingmpislurmhpc

HPC cluster : run a tool with multithread option


I'm new to the slurm job manager. I want to run the tool ADMIXTURE, which has an option to run in multithreaded mode. I don't really understand which option I have to specify in my sbatch script. This is my code :

#!/bin/bash -l
#SBATCH -A snic2022-22-1048
#SBATCH -p core
#SBATCH --mem-per-cpu=6400
#SBATCH -n 1
#SBATCH --cpus-per-task=8
#SBATCH -t 120:59:59
#SBATCH -J ADMIXTURE

for k in {2..16}
do
    mkdir "admixk${k}"
    for i in {1..10}
    do
        admixture --cv=10 -j8 merged_dataset_light.bed $k \
            > admixk${k}/struct_k${k}_${i}.out
    done
done

Where -j8 is the option which indicates that we want to run the tool with 8 threads. The documentation of ADMIXTURE says the following :

2.11 Multithreaded mode
To split ADMIXTURE’s work among N threads, you may append the flag -jN to your ADMIXTURE command. The core algorithms will run up to N times as fast, presuming you have at least N processors.

I don't know then if I have to specify

#SBATCH -n 1
#SBATCH --cpus-per-task=8

like I did or

#SBATCH -n 8
#SBATCH --cpus-per-task=1

Do you have any idea of how I should do that ? I saw this answer that says it depends of which type of parallelism is used, but how can I know in this case ?


Solution

  • In this case you should use

    #SBATCH -n 1
    #SBATCH --cpus-per-task=8
    

    With the other option each task may be allocated on another node, which obviously wouldn't work with threads.