simple python program with multiprocessing pathos library

Question

Followup with this question pathos multiprocessing cannot pickle

Running the code in one:

from pathos.multiprocessing import ProcessingPool as Pool

def f(x):
    return x*x

if __name__ == '__main__':
    p = Pool(5)
    print(p.map(f, [1,2,3]))

Traceback (most recent call last):
  File "\\femto.niddk.nih.gov\C\All Projects\NMR High Pressure\Software\multiprocessing\multiprocessing_playground_2.py", line 12, in <module>
    results = ProcessingPool().map(b.boo, [[12,3,456],[8,9,10],['a','b','cde']])
  File "C:\Python27\lib\site-packages\pathos\multiprocessing.py", line 137, in map
    return _pool.map(star(f), zip(*args)) # chunksize
  File "C:\Python27\lib\site-packages\multiprocess\pool.py", line 251, in map
    return self.map_async(func, iterable, chunksize).get()
  File "C:\Python27\lib\site-packages\multiprocess\pool.py", line 567, in get
    raise self._value
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed
>>> 

Here is some more imformation:

>>> pathos.__version__
'0.2.2.dev0'
>>> import multiprocess
>>> multiprocess.__version__
'0.70.5'
>>> import multiprocessing
>>> multiprocessing.__version__
'0.70a1'
>>> multiprocess.Pool().map(lambda x:x*x, range(10))

Traceback (most recent call last):
  File "<pyshell#11>", line 1, in <module>
    multiprocess.Pool().map(lambda x:x*x, range(10))
  File "C:\Python27\lib\site-packages\multiprocess\pool.py", line 251, in map
    return self.map_async(func, iterable, chunksize).get()
  File "C:\Python27\lib\site-packages\multiprocess\pool.py", line 567, in get
    raise self._value
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

What is going on? Why isn't working? Isn't Pathos library supposed to use dill instead of pickling?

Answer 1

it works if I use

from pathos.pools import ParallelPool as Pool

instead of

from pathos.multiprocessing import ProcessingPool as Pool

of

from pathos.pools import ProcessPool as Pool

Who can explain what is the difference? All of these methods create different processes.