I thought about a interesting question and I hope somebody can help me solve this! I want to use multiprocessing, so I choose to use pool.starmap(myfunction,zip([1,2,3,4,5],['a','b','c','d','e'])) in order to pass multi arguments. I want to combine the entire list [1,2,3,4,5] with the every single element in the second list such as
([1,2,3,4,5],'a'),([1,2,3,4,5],'b').....
instead of only combining the single element in the lists such as
(1,'a'),(2,'b')
I know how to do it in a stupid way which is multiply the list by 5
new_list=[1,2,3,4,5]*5
and then zip the new_list with the second list
I'm now wondering if there is a better way to do this?
After reading your comment I assume you are looking for itertools.repeat:
import itertools
import multiprocessing
def combine(val, char):
return f'{val}{char}'
vals = [1, 2, 3, 4, 5]
chars = ['a', 'b', 'c', 'd', 'e']
pool = multiprocessing.Pool(3)
combs = pool.starmap(combine, zip(itertools.repeat(vals, 5), chars))
print(combs)
This has a smaller memory footprint than the naive approach, which is simply
combs = pool.starmap(combine, zip([vals]*5, chars))
If you instead want to generate all combinations of the valsand the chars elements, you could use itertools.product (which is what I first assumed you wanted):
combs = pool.starmap(combine, itertools.product(vals, chars))
As a sidenote; itertools also contain a starmap function that works more or less the same as the multiprocessing one, except for executing all calls in one process, in order. This, however, can not take advantage of multiple cores.