I'm building an application that let's the user export his/her work. This is a computationally heavy process, lasting for a minute or so, during which I want to show a progress bar (and make the rest of the UI unresponsive).
I've tried the implementation below, which works fine for a non-computationally expensive background process (e.g. waiting for 0.1 s). However, for a CPU heavy process, the UI becomes very laggy and unresponsive (but not completely unresponsive).
Any idea how I can solve this?
import sys
import time
from PySide2 import QtCore
from PySide2.QtCore import Qt
import PySide2.QtWidgets as QtWidgets
class MainWindow(QtWidgets.QMainWindow):
"""Main window, with one button for exporting stuff"""
def __init__(self, parent=None):
super().__init__(parent)
central_widget = QtWidgets.QWidget(self)
layout = QtWidgets.QHBoxLayout(self)
button = QtWidgets.QPushButton("Press me...")
button.clicked.connect(self.export_stuff)
layout.addWidget(button)
central_widget.setLayout(layout)
self.setCentralWidget(central_widget)
def export_stuff(self):
"""Opens dialog and starts exporting"""
some_window = MyExportDialog(self)
some_window.exec_()
class MyAbstractExportThread(QtCore.QThread):
"""Base export thread"""
change_value = QtCore.Signal(int)
def run(self):
cnt = 0
while cnt < 100:
cnt += 1
self.operation()
self.change_value.emit(cnt)
def operation(self):
pass
class MyExpensiveExportThread(MyAbstractExportThread):
def operation(self):
"""Something that takes a lot of CPU power"""
some_val = 0
for i in range(1000000):
some_val += 1
class MyInexpensiveExportThread(MyAbstractExportThread):
def operation(self):
"""Something that doesn't take a lot of CPU power"""
time.sleep(.1)
class MyExportDialog(QtWidgets.QDialog):
"""Dialog which does some stuff, and shows its progress"""
def __init__(self, parent=None):
super().__init__(parent, Qt.WindowCloseButtonHint)
self.setWindowTitle("Exporting...")
layout = QtWidgets.QHBoxLayout()
self.progress_bar = self._create_progress_bar()
layout.addWidget(self.progress_bar)
self.setLayout(layout)
self.worker = MyInexpensiveExportThread() # Works fine
# self.worker = MyExpensiveExportThread() # Super laggy
self.worker.change_value.connect(self.progress_bar.setValue)
self.worker.start()
self.worker.finished.connect(self.close)
def _create_progress_bar(self):
progress_bar = QtWidgets.QProgressBar(self)
progress_bar.setMinimum(0)
progress_bar.setMaximum(100)
return progress_bar
if __name__ == "__main__":
app = QtWidgets.QApplication()
window = MainWindow()
window.show()
sys.exit(app.exec_())
Thanks oetzi. This works better, but still drags the UI down somewhat. I did some research, and found the following, for those who are interested.
The difficulty with showing a responsive user-interface while running a computationally heavy process using threading, stems from the fact in this case one combines a so-called IO-bound thread (i.e. the GUI), with a CPU-bound thread (i.e. the computation). For a IO-bound process, the time it takes to complete is defined by the fact that the thread has to wait on input or output (e.g. a user clicking on things, or a timer). By contrast, the time required to finish a CPU-bound process is limited by the power of the processing unit performing the process.
In principle, mixing these types of threads in Python should not be a problem. Although the GIL enforces that only one thread is running at a single instance, the operating system in fact splits the processes up into smaller instructions, and switches between them. If a thread is running, it has the GIL and executes some of its instructions. After a fixed amount of time, it needs to release the GIL. Once released, the GIL can schedule activate any other 'runnable' thread - including the one that was just released.
The problem however, is with the scheduling of these threads. Here things become a bit fuzzy for me, but basically what happens is that the CPU-bound thread seems to dominate this selection, from what I could gather due to a process called the "convey effect". Hence, the erratic and unpredictable behavior of a Qt GUI when running a CPU-bound thread in the background.
I found some interesting reading material on this:
So... this is very nice and all, how do we fix this?
In the end, I managed to get what I want using multiprocessing. This allows you to actually run a process parallel to the GUI, instead in sequential fashion. This ensures the GUI stays as responsive as it would be without the CPU-bound process in the background.
Multiprocessing has a lot of difficulties of its own, for example the fact that sending information back and forth between processes is done by sending pickled objects across a pipeline. However, the end-result is really superior in my case.
Below I put a code snippet, showing my solution. It contains a class called ProgressDialog, which provides an easy API for setting this up with your own CPU-bound process.
"""Contains class for executing a long running process (LRP) in a separate
process, while showing a progress bar"""
import multiprocessing as mp
from PySide2 import QtCore
from PySide2.QtCore import Qt
import PySide2.QtWidgets as QtWidgets
class ProgressDialog(QtWidgets.QDialog):
"""Dialog which performs a operation in a separate process, shows a
progress bar, and returns the result of the operation
Parameters
----
title: str
Title of the dialog
operation: callable
Function of the form f(conn, *args) that will be run
args: tuple
Additional arguments for operation
parent: QWidget
Parent widget
Returns
----
result: int
The result is an integer. A 0 represents successful completion, or
cancellation by the user. Negative numbers represent errors. -999
is reserved for any unforeseen uncaught error in the operation.
Examples
----
The function passed as the operation parameter should be of the form
``f(conn, *args)``. The conn argument is a Connection object, used to
communicate the progress of the operation to the GUI process. The
operation can pass its progress with a number between 0 and 100, using
``conn.send(i)``. Once the process is finished, it should send 101.
Error handling is done by passing negative numbers.
>>> def some_function(conn, *args):
>>> conn.send(0)
>>> a = 0
>>> try:
>>> for i in range(100):
>>> a += 1
>>> conn.send(i + 1) # Send progress
>>> except Exception:
>>> conn.send(-1) # Send error code
>>> else:
>>> conn.send(101) # Send successful completion code
Now we can use an instance of the ProgressDialog class within any
QtWidget to execute the operation in a separate process, show a progress
bar, and print the error code:
>>> progress_dialog = ProgressDialog("Running...", some_function, self)
>>> progress_dialog.finished.connect(lambda err_code: print(err_code))
>>> progress_dialog.open()
"""
def __init__(self, title, operation, args=(), parent=None):
super().__init__(parent, Qt.WindowCloseButtonHint)
self.setWindowTitle(title)
self.progress_bar = QtWidgets.QProgressBar(self)
self.progress_bar.setValue(0)
layout = QtWidgets.QHBoxLayout()
layout.addWidget(self.progress_bar)
self.setLayout(layout)
# Create connection pipeline
self.parent_conn, self.child_conn = mp.Pipe()
# Create process
args = (self.child_conn, *args)
self.process = mp.Process(target=operation, args=args)
# Create status emitter
self.progress_emitter = ProgressEmitter(self.parent_conn, self.process)
self.progress_emitter.signals.progress.connect(self.slot_update_progress)
self.thread_pool = QtCore.QThreadPool()
def slot_update_progress(self, i):
if i < 0:
self.done(i)
elif i == 101:
self.done(0)
else:
self.progress_bar.setValue(i)
def open(self):
super().open()
self.process.start()
self.thread_pool.start(self.progress_emitter)
def closeEvent(self, *args):
self.progress_emitter.running = False
self.process.terminate()
super().closeEvent(*args)
class ProgressEmitter(QtCore.QRunnable):
"""Listens to status of process"""
class ProgressSignals(QtCore.QObject):
progress = QtCore.Signal(int)
def __init__(self, conn, process):
super().__init__()
self.conn = conn
self.process = process
self.signals = ProgressEmitter.ProgressSignals()
self.running = True
def run(self):
while self.running:
if self.conn.poll():
progress = self.conn.recv()
self.signals.progress.emit(progress)
if progress < 0 or progress == 101:
self.running = False
elif not self.process.is_alive():
self.signals.progress.emit(-999)
self.running = False