Search code examples
pythonthreadpoolexecutor

Executor Map doesn't run a function


I'm trying to reproduce a race condition. The code below works fine with submit used instead of map but I want to understand why map doesn't run the update method.

import concurrent.futures
import time


class BankAccount:
    def __init__(self):
        self.balance = 100  # shared data

    def update(self, transaction, amount):
        print(f'{transaction} started')
        tmp_amount = self.balance  # reading from db
        tmp_amount += amount
        time.sleep(1)
        self.balance = tmp_amount
        print(f'{transaction} ended')


if __name__ == '__main__':
    acc = BankAccount()
    print(f'Main started. acc.Balance={acc.balance}')

    with concurrent.futures.ThreadPoolExecutor(max_workers=2) as ex:
        ex.map(acc.update, [dict(transaction='refill', amount=100),
                            dict(transaction='withdraw', amount=-200)])

    print(f'End of Main. Balance={acc.balance}')

Solution

  • Pass each kind of argument in a separate iterable.

    ex.map(acc.update, ['refill', 'withdraw'], [100, -200])
    

    Like map, Executor.map takes one iterable per argument, and assigns from each iterable to one argument. In addition, errors are not propagated until the result is actually accessed.

    Executor.map(func, *iterables, timeout=None, chunksize=1)

    Similar to map(func, *iterables) ...

    If a func call raises an exception, then that exception will be raised when its value is retrieved from the iterator.

    Thus, the code incorrectly passes the arguments, but suppresses the error. Fetching the values, e.g. via list(ex.map(...)) reveals the error:

    TypeError: update() missing 1 required positional argument: 'amount'
    

    Creating a separate iterable for each kind of argument prevents this.

    #                  V transaction
    ex.map(acc.update, ['refill', 'withdraw'], [100, -200])
    #                                          ^ amount
    

    It may be desirable to order arguments by call, not by kind. Use zip and * unpacking to convert the input as required by map.

    ex.map(acc.update, *zip(('refill', 100), ('withdraw', 200)))
    

    If keyword arguments are desired, a helper is required to unpack the arguments.

    def kwargify(func):
        """Wrap ``func`` to accept a dictionary of keyword arguments"""
        return lambda kwargs: func(**kwargs)
    

    Simply wrap the desired function with this helper when passing it to ex.map:

        ex.map(
            kwargify(acc.update),
            [
                dict(transaction='refill', amount=100),
                dict(transaction='withdraw', amount=-200)
            ]
        )