Pythons multiprocess module (with dill) gives an unhelpful AssertionError

Question

I have installed dill/pathos and its dependencies (with some difficulty) and I'm trying to perform a function over several processes. The class/attribute Model(self.xml,self.exp_data,i).SSR is custom made and depends on loads of other custom functions so I apologize in advance for not being able to provide 'runnable' code. In brief however, it takes some experimental data, integrates a system of ODE's with python's pysces module and calculates the sum of squares (SSR). The purpose for parallelizing this code is to speed up this calculation with multiple parameter sets.

The code:

    import multiprocess  
    def evaluate_chisq(pop):

        p = multiprocess.Pool(8)
        res= p.map(lambda i:Model(self.xml,self.exp_data,i).SSR   ,  pop)#calcualteSSR with this parameter set
        return res

The error message I get is:

  File "C:\Anaconda1\lib\site-packages\multiprocess\pool.py", line 251, in map
    return self.map_async(func, iterable, chunksize).get()

  File "C:\Anaconda1\lib\site-packages\multiprocess\pool.py", line 567, in get
    raise self._value

AssertionError

Then I have tried using map_async :

        def evaluate_chisq(pop):
            p = multiprocess.Pool(8)
            res= p.map_async(lambda i:Model(self.xml,self.exp_data,i).SSR   ,  pop)#calcualteSSR with this parameter set
            return res

which returns a <multiprocess.pool.MapResult object at 0x0000000014AF8C18> object which gives me the same error when I attempts to use the MapResult's `get' method

  File "C:\Anaconda1\lib\site-packages\multiprocess\pool.py", line 567, in get
    raise self._value

AssertionError

Does anybody know what I'm doing wrong?

Answer 1

On Windows you need to use freeze_support from __main__.

See https://docs.python.org/2/library/multiprocessing.html#multiprocessing.freeze_support.