Basic parallelization. Create a pool of workers and all of them run the same function in parallel

Question

I have the following code

class calculator:
    def __init__(self, value):
      print("hi")
      result = self.do_stuff()
      print(result)
if __name__ == '__main__':
  calculator(20) # I want to do this 4 times in parallel

I have a class that calculates stuff. I have 4 processors. So I want to instantiate this class 4 times with value 20. These 4 identical classes with the same input then calculate something in parallel. To make sure that it works asynchronly I want to see in the console 4 times hi and after a short while 4 times the result.

But I don't really know how to do that with

 import multiprocessing as mp

It seems that you can only parallelize definitions so I added the definition

def start_calculator(value):
    calculator(value)

and the main is now

if __name__ == '__main__':
p1 = mp.Process(target=start_calculator, args=(20,))
p2 = mp.Process(target=start_calculator, args=(20,))
p3 = mp.Process(target=start_calculator, args=(20,))
p4 = mp.Process(target=start_calculator, args=(20,))
p1.start()
p2.start()
p3.start()
p4.start()
p1.join()
p2.join()
p3.join()
p4.join()

but this just looks incredibly bad.

can I somehow create 4 processes and then in a loop start them all and then join them all without writing p1,p2,p3,p4 every time. And also I want to make it variable to only have 2 Processes etc. So I can't hardcode it like this

Ideally I would have an array with a fixed amount of processes and then I give them asynchronally the function. And If one process is finished I would be able to give it the function again

Answer 1

Can you use the pool?

pool = mp.Pool(4) #with no argument it takes the maximum
args = [20.,20.,20.,20.]
output = pool.map(start_calculator,args)
pool.close()
pool.join()