How to tell Pool to use a certain number of cores per process?

I'm using multiprocessing.Pool to parallelise some computation in a project. How can I tell Pool to use n (eg. 4) cores per parallel process?

Say I have 8 cores. Will this code make sure that each parallel process is running on 4 cores?

from multiprocessing import Pool

def fun(in):
    print(in)

pool = Pool(2)
pool.map(fun, [1, 2, 3, 4, 5, 6])

Solution

multiprocessing.Pool will not create multiple threads per process, but multiple single-threaded processes. "Parallel processes" means multiple processes that run in parallel, not single processes that are somehow internally parallel.

Each process in a multiprocessing.Pool will run on exactly one core at a time, so you should create as many processes as you want cores to be utilised - in this case if you want to potentially utilise all eight cores, you want eight processes in the pool:

pool = Pool(8)

You can also not pass an argument at all, and Pool will automatically allocate as many processes as you have CPU cores.

Documentation for multiprocessing.Pool:

processes is the number of worker processes to use. If processes is None then the number returned by os.cpu_count() is used.

Note, however, that you cannot actually tell Pool to use a specific core or specific amount of cores - that decision is made by your operating system, which will usually try to allocate workloads evenly across cores.