Search code examples
pythonmultiprocessingpython-multiprocessingcpu-cores

How to identify the CPU core ID of a process on Python multiprocessing?

I am testing Python's multiprocessing module on a cluster with SLURM. I want to make absolutely sure that each of my tasks are actually running on separate cpu cores as I intend. Due to the many possibilities of configuring SLURM, this is not at all obvious.

Therefore, I was wondering if there is a way to get core specific information from the running Python task. I need my Python script to get information about the core it's running on which allows to distinguish between the various cores.

Consider the following script. There are 10 tasks launched. Is there a way for each of them to print core specific information about the core they have been assigned to? 

import multiprocessing

def hello():
    print "Hello World"

pool = multiprocessing.Pool()
jobs = []
for j in range(10):
    p = multiprocessing.Process(target = hello)
    jobs.append(p)
    p.start()

The modules I found here and here give CPU specific information, but these are about available CPUs, not core information at runtime.

Solution

  • On Linux, FreeBSD, SunOS you can use the psutil-module for that:

    psutil.Process().cpu_num()

    Note, that as long as you don't set the cpu-affinity as well, your OS will arbitrarily migrate the execution to different cores. Processes are not assigned to specific cores by default and it's usually best to let your OS decide on which core to run something.