Search code examples
pythonpython-multiprocessing

Why pool class in multiprocessing is not giving advantage over linear processing


I am trying to measure the advantage of pool class in multiprocessing module over normal programming and I am calculating square of a number using a function . Now when I calculate the time take to find the square of all the three numbers it takes around ~0.24 sec but when I calculate it normally in a for loop it takes even less ~0.007 sec. Why is that? Shouldn't the code part with pool should be faster?

import time

from multiprocessing import Pool,Process

def f(x):
   return x*x

if __name__ == '__main__':

   start = time.time()
   array = []
   for i in range(1000000):
       array.append(i)

   with Pool(4) as p:
       (p.map(f, array))

   print(time.time()-start)  # time taken when using pool

   start1 = time.time()



   for i in range(1000000):
       f(array[i])



   print(time.time()-start1)   # time taken normaly




Solution

  • So as suggested by @klaus D. and @wwii I was not having enough computation to overcome the overhead of spawning processes and the time taken in switching between the processes. Below is the updated code to notice the difference. Hope it helps

    import multiprocessing
    import time
    import random
    from multiprocessing import Pool,Process
    
    def f(x):
        time.sleep(3)
    
    if __name__ == '__main__':
    
    
        array = []
    
        for i in range(4):
            array.append(i)
    
        start = time.time()
    
        with Pool(4) as p:
            (p.map(f, array))
    
        print(time.time()-start)  # time taken when using pool
    
        start1 = time.time()
    
    
    
        for i in range(4):
            f(array[i])
    
    
    
        print(time.time()-start1)   # time taken normaly