Why does Pythons ThreadPoolExecutor supress all Exceptions and how to fix this?

I am relatively new to Python, and I am having difficulty understanding the behavior of this pool. I have attempted to view similar questions on stackoverflow, but I am still having difficulties on comprehending this problem.

For example if we have use of an Executor: with ThreadPoolExecutor(8) as pool: pool.map(doSomething,test_list)

All errors that occur within the pool are suppressed and not shown.

Is there any parameter available so the pool stops execution and shows the exceptions?

Thanks in advance!

Solution

I have found a possibility for myself which can circumvent this problem indirectly. It's actually pretty simple so I'd like to publish it here if someone else encounters this problem. I do not know if it's very clever, but for me this solution was fine.

At first thanks to @artiom, as he already said, it is best to catch the error directly in the parallelized function.

Now the "bypass":

Put a try except block inside the function around the whole code and catch all exceptions with

except Exception as e:

. The exception is simply logged in the except block, for example with sentry or another logging framework. With that, you can at least look at the thrown exceptions again and fix your mistakes in your code.

Simple example:

def for_parallel(chunk):
    for one in chunk:
        try:
            a = 1
            foo(one['bla'])
            doSomething(one['blu'])
        except Exception as e:
            sentry.capture_exception(e)


def main():
    with ThreadPoolExecutor(8) as pool:
        pool.map(for_parallel, my_list)