Search code examples
pythonmultithreadingpython-3.xpython-multithreadingconcurrent.futures

How to use concurrent.futures in Python


Im struggling to get multithreading working in Python. I have i function which i want to execute on 5 threads based on a parameter. I also needs 2 parameters that are the same for every thread. This is what i have:

from concurrent.futures import ThreadPoolExecutor

def do_something_parallel(sameValue1, sameValue2, differentValue):
    print(str(sameValue1))        #same everytime
    print(str(sameValue2))        #same everytime
    print(str(differentValue))    #different

main(): 

    differentValues = ["1000ms", "100ms", "10ms", "20ms", "50ms"]

    with ThreadPoolExecutor(max_workers=5) as executor:
        futures = [executor.submit(do_something_parallel, sameValue1, sameValue2, differentValue) for differentValue in differentValues]

But i don't know what to do next


Solution

  • If you don't care about the order, you can now do:

    from concurrent.futures import as_completed
    
    # The rest of your code here
    
    for f in as_completed(futures):
        # Do what you want with f.result(), for example:
        print(f.result())
    

    Otherwise, if you care about order, it might make sense to use ThreadPoolExecutor.map with functools.partial to fill in the arguments that are always the same:

    from functools import partial
    
    # The rest of your code...
    
    with ThreadPoolExecutor(max_workers=5) as executor:
        results = executor.map(
            partial(do_something_parallel, sameValue1, sameValue2),
            differentValues
        ))