Multiprocessing Error with Results

Question

I have a small problem in a big program, so I made a small example, which shows my problem:

import multiprocessing

class class1():
    def classfunction1(self, a):
        self.x = a

class class2():
    def classfunction2(self, a):
        self.y = a

def test(i):
    print("I'm in the Testfunction")
    b = i * class1.x * class2.y

    return b

class1 = class1()
class2 = class2()

if __name__ == "__main__":
    x = 1
    y = 2
    class1.classfunction1(x)
    class2.classfunction2(y)
    print("This variable is callable", class1.x)
    print("And this one is also callable", class2.y)
    counter = []
    for i in range(10):
        counter.append(i)
    pool = multiprocessing.Pool(processes=4)
    results = pool.imap(test, counter)
    pool.close()
    pool.join()
    #resultslist = list(results)

When I use the last line resultslist = list(results) I get the follow Error:

multiprocessing.pool.RemoteTraceback:

Traceback (most recent call last):
  File "C:\Program Files\Anaconda3\lib\multiprocessing\pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "C:\Users\example2.py", line 24, in test
    b = i * class1.x * class2.y
AttributeError: 'class1' object has no attribute 'x'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\example2.py", line 43, in <module>
    resultslist = list(results)
  File "C:\Program Files\Anaconda3\lib\multiprocessing\pool.py", line 695, in next
    raise value
AttributeError: 'class1' object has no attribute 'x'

It is necessary that the commands class1.classfunction1(x) and class2.classfunction2(y) are in the if__name__=="__main__". I need the basic structure of this script, so please don't do too many changes (if possible).

Answer 1

The problem is you're trying to access the x attribute of the class1 instance of class1(!) that has never been created in the worker subprocesses because class1.classfunction1() isn't called in them.

To fix that issue I added an init() function and call it both in the main process and specify it as the initializer function for each of the subprocesses created by the Pool:

import multiprocessing

class class1():
    def classfunction1(self, a):
        self.x = a

class class2():
    def classfunction2(self, a):
        self.y = a

def test(i):
    print("I'm in the Testfunction")
    b = i * class1.x * class2.y

    return b

def init():  # added
    global class1, class2
    class1 = class1()
    class2 = class2()
    x = 1
    y = 2
    class1.classfunction1(x)
    class2.classfunction2(y)

if __name__ == "__main__":
    init()  # explicit call here
    print("This variable is callable", class1.x)
    print("And this one is also callable", class2.y)
    counter = []
    for i in range(10):
        counter.append(i)
    pool = multiprocessing.Pool(initializer=init, processes=4)  # implicit call
    results = pool.imap(test, counter)
    pool.close()
    pool.join()
    resultslist = list(results)
    print(resultslist)