Search code examples
pythonpython-3.xmultithreadingconcurrencyconcurrent.futures

How can I get the arguments I sent to ThreadPoolExecutor when iterating through future results?


I use a ThreadPoolExecutor to quickly check a list of proxies to see which ones are dead or alive.

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    futures = []
    for proxy in proxies:
        future = executor.submit(is_proxy_alive, proxy)
        futures.append(future)
    
    for future in futures:
        print(future.result()) # prints true or false depending on if proxy is alive.
                               # how do I get the specific proxy I passed in the arguments 
                               # so that I can make a dictionary here?

My goal is to get the argument (the proxy) I passed to the executor when iterating through the results to know which exact proxies are dead or alive, so I could make a dictionary that might look like this:

{"IP1": False, "IP2": True, "IP3": True}

One method I can think of is returning the proxy I sent on top of returning true/false, but is there a better way to do it externally so the function doesn't have to return more than just a bool?


Solution

  • While submitting the task, you could create a mapping from future to its proxy.

    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
        future_proxy_mapping = {} 
        futures = []
        for proxy in proxies:
            future = executor.submit(is_proxy_alive, proxy)
            future_proxy_mapping[future] = proxy
            futures.append(future)
        
        for future in futures:
            proxy = future_proxy_mapping[future]
            print(proxy)
            print(future.result())