Search code examples
pythonthreadpoolpython-multithreadingpool

How to kill a Threading pool from parent?


I have threading classing that has the following run function. So when this class is set to run it keeps on checking a multiprocessing manager queue, if there is anything inside it, it starts the pool to run the job(track function). Upon completion of the job, pool closes automatically and the whole queue if not empty check starts.

def runQueue(self):
    print("The current thread is", threading.currentThread().getName())
    while True:
        time.sleep(1)
        self.pstate=False
        if self.runStop: #this stops the whole threading by dropping main loop
            break
        while not self.tasks.empty():
            self.pstate=True
            task = self.tasks.get()
            with ThreadPool(processes=1) as p: #<- want to kill this pool
                ans = p.apply(self.track, args=(task,))
                self.queueSend(ans)
                self.tasks.task_done()
    print("finished job")

I used the pool because the function returns a value which I need to map. What I am looking for is a way such that, upon some parent call, the pool closes by dropping the job, while keeping the primary class thread (run function [main loop] running). Any kind of help is appreciated.


Solution

  • I found that for my case pool.terminate would work only I/O applications, I did find some solutions online which were not related to the pool but I could implement. One solution is to run the thread as a multiprocessing process and then call process.terminate() or using multiprocessing Pool and then call pool.terminate.
    Note that multiprocessing is faster for CPU intensive tasks. If the tasks are I/O intensive threads are the best solution.
    The only way I found a way to kill the thread is using win32 ctypes module. If you start a thread and get it's tid with

    tid thread.ident
    

    then you can put your in kill_thread(tid) function below

    w32 = ctypes.windll.kernel32
    THREAD_TERMINATE = 1 
    def kill_thread(tid):
            handle = w32.OpenThread(THREAD_TERMINATE, False,tid)
            result = w32.TerminateThread(handle, 0)
            w32.CloseHandle(handle)
    

    Hope this helps someone.