Search code examples
pythonmultithreadingthreadpoolexecutor

Doesn't ThreadPoolExecutor start a new thread before the last thread is finished


I have written down this on Colab

import threading
import time
import os

class thread():
    def __init__(self, url, id, sleep_time):
        self.url = url
        self.id = id
        self.sleep_time = sleep_time

    def run(self, key):
        print("Task Executed {}".format(threading.current_thread()))
        self.upload_image(key)

    def upload_image(self, key):
        for i in range(10):
          print(self.id, "->", key)
          print()
          time.sleep(10)

with ThreadPoolExecutor(max_workers=1000) as executor:
  for i in range(1000):
    t = thread("url", i, 0.5)
    print("start the id: ", i)
    executor.submit(t.run("gg"))

Whenever the last thread is run, it is going to sleep for 10 seconds before it executes for the next print. However, the program doesn't start a new thread when the previous one is running. For example, if thread 0 is running, and it sleeps for 10 seconds, shouldn't the program start thread 1 when executing thread 0?


Solution

  • You don't tell the executor to run the method, you're executing yourself.

    executor.submit(t.run("gg"))
    

    This will evaluate t.run("gg") - i.e execute the function, before it is even submitted.

    You need to tell the executor what to execute, but not execute it:

    executor.submit(t.run, ("gg",))
    

    Here I'm telling it to do t.run with the arguments ("gg",)

    Which results in:

    start the id:  0
    Task Executed <Thread(ThreadPoolExecutor-0_0, started 123145503694848)>
    0 -> ('gg',)
    
    start the id:  1
    Task Executed <Thread(ThreadPoolExecutor-0_1, started 123145520484352)>
    1 -> ('gg',)
    ...