Search code examples
pythonpython-multiprocessing

Is it possible to do multiprocessing within a class method, and obtain the returning values in an order?


I am relatively new to python and especially its multiprocessing capabilities. Apparently in python i read that to have proper multiprocessing is using its multiprocessing library which actually creates new python processes unlike others which divide into threads within the same core.

I came across an example which uses the Pool object with its method map for this, but when i tried to implement it on a method within a class, it seems that the spawnpoolworker cannot reach the method into the class.

For example, the following code gives the error AttributeError: 'A' object has no attribute '__multiply'.

import multiprocessing as mp
import time

class A:
    def __init__(self,data):
        self.__data = data
        self.sum = [0]*len(data)
        self.__multiprocess()

    def __multiply(self,num):
        time.sleep(1)
        return num*2

    def __multiprocess(self):
        pool = mp.Pool()
        results = pool.map(self.__multiply,self.__data)

        for i, result in zip(len(self.__data), results):
            self.sum[i] = result

if __name__ == "__main__":
    data = list(range(0,8))
    a = A(data)
    print(a.sum)

Is there a way that lets me keep the structure of the class without splitting it into functions while giving the result in order of the data given?

print(a.sum)
[0, 2, 4, 6, 8, 10, 12, 14]

Solution

  • You can use this code block. The problem was using __ (private functions)

    import multiprocessing as mp
    import time
    
    class A:
         def __init__(self,data):
             self.__data = data
             self.sum = [0]*len(data)
             self.multiprocess()
    
         def multiply(self,num):
             time.sleep(1)
             return num*2
    
         def multiprocess(self):
             pool = mp.Pool()
             results = pool.map(self.multiply,self.__data)
    
             for i, result in zip(range(len(self.__data)), results):
                 self.sum[i] = result
    
    if __name__ == "__main__":
         data = list(range(0,8))
         a = A(data)
         print(a.sum)
    [0, 2, 4, 6, 8, 10, 12, 14]