Search code examples
pythonconcurrencyiteratorfuturemap-function

Pass multiple parameters to concurrent.futures.Executor.map?


The concurrent.futures.Executor.map takes a variable number of iterables from which the function given is called. How should I call it if I have a generator that produces tuples that are normally unpacked in place?

The following doesn't work because each of the generated tuples is given as a different argument to map:

args = ((a, b) for (a, b) in c)
for result in executor.map(f, *args):
    pass

Without the generator, the desired arguments to map might look like this:

executor.map(
    f,
    (i[0] for i in args),
    (i[1] for i in args),
    ...,
    (i[N] for i in args),
)

Solution

  • You need to remove the * on the map call:

    args = ((a, b) for b in c)
    for result in executor.map(f, args):
        pass
    

    This will call f, len(args) times, where f should accept one parameter.

    If you want f to accept two parameters you can use a lambda call like:

    args = ((a, b) for b in c)
    for result in executor.map(lambda p: f(*p), args):   # (*p) does the unpacking part
        pass