Search code examples
pythonpathos

Run two functions in parallel in pathos


I have 2 different functions, let's say:

def foo(print_me):   
    print(print_me + " foo")

def foo2(print_me):   
    print(print_me + " foo2")

I want to run them both, parallel, in pathos.

in multiprocessing I will do something like:

process = [Process(target=foo, args=("HI")),Process(target=foo2, args=("HI2")]
map(lambda p: p.start(), process)
map(lambda p: p.join(), process)

How can I do something similar in pathos?

P.S
I can't use python multiprocessing because in the real functions, I am using spacy which can't use multiprocessing (due to the pickle error).


Solution

  • Like this:

    >>> def foo(print_me):   
    ...     print(print_me + " foo")
    ... 
    >>> def foo2(print_me):   
    ...     print(print_me + " foo2")
    ... 
    >>> from pathos.helpers import mp
    >>> process = [mp.Process(target=foo, args=("HI",)),mp.Process(target=foo2, args=("HI2",))]
    >>> r1 = map(lambda p: p.start(), process) 
    >>> r2 = map(lambda p: p.join(), process) 
    >>> r1 = list(r1); r1 = list(r2)
    HI foo
    HI2 foo2
    >>> 
    

    I also corrected the typos in your code above, but not in your question.