I'm working on a simple script that utilizes the multiprocessing module. I have no experience with this module and I'm just trying to understand how to use it. I'm starting simple. However, I've encountered a puzzling behavior where the print statement in my script is being executed twice.
My understanding is that, as long as I'm running the script directly, the code under if __main__
should run. And, the entire script should run procedurally. I was expecting the print statement to run only once prior to the if
block. Why is it being executed multiple times? I typically don't structure my code using this pattern, so I'm wondering if there's behavior associated with using if __main__
that I might not be aware of. It's just a normal if
statement like any other."
I'm not sure what's going on here. Can someone please provide some insights into why this might be happening? Thank you! I'm running python 3.11.3 from Anacaonda-Spyder 5.4.3
import multiprocessing
print("TEST TEST TEST")
def square(x):
return x * x
if __name__ == '__main__':
numbers = [1, 2, 3, 4, 5]
with multiprocessing.Pool() as pool:
results = pool.map(square, numbers)
print(results)
Results:
TEST TEST TEST
[1, 4, 9, 16, 25]
TEST TEST TEST
TEST TEST TEST
See Context and start methods. One print is coming from your top-level processes, and two from your worker processes.
When using the spawn method, multiprocessing may import your module to access the square
function. If you don't want module level code such as this print statement to be executed, it needs to go inside the if __name__ == '__main__':
guard.
If you set the start method to fork, then you should only get one print here:
if __name__ == '__main__':
multiprocessing.set_start_method('fork')
...
But don't rely on that - instead fix your module so that the import doesn't have side-effects.