Search code examples
pythonfunctionmultiprocessingreturnpool

Returning values from multiprocessing Pool function


I want to run a loop in parallel using pool and store each result from a return of a function into an index of numpy array. I have written a basic function here, real one is a bit complex. Even in this basic one I am not getting desired output. By printing results at the end I am getting 100 different arrays of 100 values instead of one array of 100 values. How do I solve this or is there a better way to store return values. Because I have to take a mean and std of rejects after pool.


from multiprocessing import Pool
import numpy as np
rejects = np.zeros(100)

def func(i):
    print("this is:",i)
    rejects[i]=i
    # print (rejects)
    return rejects




def main():

    
    l = [*range(1,100, 1)]
    pool = Pool(3)
    results=pool.map(func, l)
    pool.close()
    pool.join()
    print (results)
if __name__ == '__main__':
    main()

Solution

  • Because you are giving an array argument to func and also assigning that array as a single element in the array rejects. you can use the func below:

    def func(i):
        print("this is:",i)
        rejects=i # this is where I have changed
        # print (rejects)
        return rejects