Making parallel code work in python 2.7 and 3.6

Question

I have some code in python 3.6 which is like this:

from multiprocessing import Pool
with Pool(processes=4) as p:
    p.starmap(parallel_function, list(dict_variables.items()))

Here dict_variables looks like this:

[('aa', ['ab', 'ab', 'ad']), ('aa1', ['a1b', 'a1b', 'a2d'])]

This code only works in python 3.6. How can I make it work in 2.7?

Answer 1

starmap was introduced in Python3.3. In Python2, use Pool.map and unpack the argument yourself:

In Python3:

import multiprocessing as mp

def starmap_func(x, y):
    return x**y

with mp.Pool(processes=4) as p:
    print(p.starmap(starmap_func, [(1,2), (3,4), (5,6)]))
    # [1, 81, 15625]

In Python2 or Python3:

import multiprocessing as mp

def map_func(arg):
    x, y = arg
    return x**y

p = mp.Pool(processes=4)
print(p.map(map_func, [(1,2), (3,4), (5,6)]))
# [1, 81, 15625]
p.close()