I'm trying to add multiple long-running threads to ThreadPool in Python3. For each thread how can I determine how many tasks are in front of it before it gets executed?
My goal is to display to the user "You have X items pending before your task". It's not a problem if it's not precise when it reaches max_workers.
with futures.ThreadPoolExecutor(max_workers=2) as executor:
future1 = executor.submit(task, 10)
future2 = executor.submit(task, 10)
future3 = executor.submit(task, 10)
# my naive failed attempt was
numOfRemainingTasks = (len(executor)-1) - executor.indexof(future3)
If I understand correctly you want something along these lines?
import concurrent.futures
class CustomExecutor(concurrent.futures.ThreadPoolExecutor):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def submit(self, *args, **kwargs):
print(f'There are {self._work_queue.qsize()} items in front of me')
return super().submit(*args, **kwargs)
def task(num):
return num + 10
with CustomExecutor(max_workers=2) as executor:
futures = (executor.submit(task, 10) for _ in range(6))
for future in concurrent.futures.as_completed(futures):
print(future.result())
There are 0 items in front of me
There are 0 items in front of me
There are 0 items in front of me
There are 1 items in front of me
There are 1 items in front of me
There are 2 items in front of me
....