Progress bar lagging with computationally intensive function on separate thread tkinter

Question

I'm trying to implement a GUI to assist some coworkers with image processing, but I am running into a problem with the Tkinter (ttk) progress bar, which keeps lagging while I execute work in another thread.

Basically, I want to have the progress bar run in indeterminate mode and bounce back and forth, as a kind of visual confirmation that things are still progressing (like the "working" circle in windows).

I set it up as follows:

# Frame/root set up (etc.) above here...

pbv = IntVar()
progressBar = ttk.Progressbar(consoleFrame, orient="horizontal", length=100, variable=pbv, mode="indeterminate")
progressBar.grid(column=1, row=1, sticky=N)
progressBar.start()

root.mainloop()

and it bounces along as intended, running nicely on the primary thread. Good so far.

Then, I initialize a separate thread and run a very computationally intensive function.

externalFunctionThread = threading.Thread(target=expensiveFunctionThread, args=[foo])
externalFunctionThread.deamon = True
externalFunctionThread.start()

With some function:

def expensiveFunctionThread(foo):
   for i in range(a_few_iterations):
      superDuperExpensiveFunction(i, foo)
      # Note: Were I to comment this out and replace it with a time.sleep(n), the bar would progress normally
      # Even a for loop that prints numbers up to some large integer would not cause lag

For context, superDuperExpensiveFunction is a call to a pyradiomics function, which is generating many texture features from some image.

What ends up happening is the bar will heavily lag, frequently locking up (not moving) while the superDuperExpensiveFunction is running, only moving after each iteration. Once the super duper expensive function thread finishes it will return to normal.

I have looked at other threads on this board (How to connect a progress bar to a function? , Tkinter: How to use threads to preventing main event loop from "freezing" , ttk progress bar freezing , etc.), but none of them help, as they are concerned with the progress bar freezing at the start, as opposed to lagging in its updates. The last one does not work (including progressBar.update()/progressBar.update_idletasks() does nothing). My fear is that it has to do with how Python handles threading (in that there is ultimately only 1 "real" thread that can do work at one time, and Python has to cycle which thread is allowed to work (the GIL or something?)).

Anyways, is there a workaround for this? Why might this be occurring?

Answer 1

Okay, so after doing some research I found out that pyradiomics has a flag that prohibits this kind of rapid task switching, and therefore threading is not a viable solution. To solve the problem, as furas recommended, I implemented multiprocessing (pooling, in fact!). This not only allowed me to have the progress bar run without any lag, but also enabled dividing up the computation across cores, speeding up my implementation greatly. The syntax is very similar to that of threading, with one main exception that caused me a lot of grief before I figured it out. Variables are not inherited by child processes! Therefore, should I want to get a value from an Entry or something, I would have to explicitly pass it in to the child process. Past that, there really aren't that many changes. Instead of threading just use the associated process thing (Process instead of Thread, multiprocessing.Queue instead of queue). No lag, and greater efficiency!