Search code examples
pythonmultiprocessingpool

Python multiprocessing.Pool workers remain idle

I am a beginner with Python. I am trying to run a simple function using multiprocessing.Pool, but when I run the program, it does not get processed. I can see that distinct processes are spawned, but that they remain idle. For example, the code below does does generate two worker processes but no output and nothing else:

import multiprocessing

def f(x):
    print("Process "+str(x))
    return True

if __name__ == '__main__':
    pool = multiprocessing.Pool(processes=2)            
    result = pool.map(f, range(1000))

I am running Python 3.9 using Microsoft Visual Studio 2019. OS is Windows 10. I encountered the same problem using Python 3.7 on this computer. I have tried the same code on another computer (Also Windows 10, VS2019) and it works fine. The difference is that this one has an i9 Processor, while the other has an i7.

I have searched online, but I have not encountered any similar issue or any fix for my problem. Does anyone have any idea why does this happen and/or possible fixes? May the type of processor have something to do with it? Thanks!

Solution

  • This is a just a guess:

    I know that this will not run under Jupyter Notebook or the interactive interpreter. See the multiprocessing docs, in particular the section:

    enter image description here

    The solution for those environments is to take your worker function, f, and place it in a separate module, for example worker.py, and to import the worker function:

    import multiprocessing
from worker import f

if __name__ == '__main__':
    pool = multiprocessing.Pool(processes=2)
    result = pool.map(f, range(1000))

    Give this a shot and let us know for others who also have Visual Studio whether this worked or not. Thanks.