QProgressDialog only shows after long-running code is finished

Question

I have a program designed according to the MVC model. My view has an open button where I can open files. The files are parsed and a lot of calculation is done with the file contents.

Now I want to display a loader to indicate that the user should wait. I am totally new to Python and Qt. I am using PyQt5 (Qt 5.6.2) with Python 3.6.2.

I added the showLoader() method to the openFiles() method of my view:

class MainWindow(QtWidgets.QMainWindow):
    def __init__(self, controller, parent = None):
        # initializing the window

    def showOpenDialog(self):
        files, filters = QtWidgets.QFileDialog.getOpenFileNames(self, 'Open file(s)', '',
                                          "Raw files (*.rw.dat);;Data files (*.dat)" + 
                                          ";;Text files (*.txt);;All files (*)")

        self.showLoader("Loading file(s)")

        self.doSomeStuffWithTheFiles(files)

        self.hideLoader()

    def showLoader(self, text):
        self._progress = QtWidgets.QProgressDialog(text, "Abort", 0, 0, self);
        self._progress.setWindowModality(QtCore.Qt.WindowModal);

This will display the Loader but it will appear after the file is loaded. Not even immediately after the file is loaded, but it will take an additional 1-2 seconds after everything is done (including some repainting of the Window)

I read a lot about threads so I assume the file parsing is blocking the progress loader which makes sense. I read that I should add the QProgressDialog to a slot (I don't really know what that is) but this doesn't help me because I want the QProgressDialog to be displayed after the QFileDialog.

I also read something about adding QtWidgets.QApplication.processEvents() to repaint the Window, but this didn't work for me (or I used it wrong).

So my questions are:

1. How do I display the QProgressDialog when I call the showLoader() method?
2. Do I have to execute my calculations and file parsing in a different thread and if have to how do I do this?
3. If I wanted to display some more information in the QProgressDialog like updating the text and the progress, how do I do this?

Further Question

The solution pointed out by @ekhumoro works fine. I see the loader and the files are parsed correctly. My problem is now that updating my existing MainWindow does not work.

After executing the code I see a little window popping up but it is disappearing instantly. (I had a problem like this and it was about the C++ garbage collector in the background of Qt. But in my understanding the layout should keep a reference to the ParsedDataWidget so this doesn't make sense for me.) Also the ParsedDataWidget is a widget which should be added to the layout "inline" and not appearing as a "window".

# a class that handles the data parsing of each file and creates an
# object that contains all the data with some methods...
class DataParser
    def __init__(self, data):
        # handle the data

# displaying the parsed data in a fancy way
class ParsedDataWidget(QtWidgets.QWidget)
    def __init__(self, data):
        # create some UI

# the worker class just like @ekhumoro wrote it (stripped down to
# relevant code)
class Worker(QtCore.QObject):
    def run(self):
        self._stop = False
        for count, file in enumerate(self._files, 1):

            # parse the data in the DataParser and create an object
            # of the files data
            data = DataParser(file)

            # works fine, the data is parsed correctly
            print(data)

            # does not work
            window.addParsedData(data)

            self.loaded.emit(count, file)
            if self._stop:
                break
        self.finished.emit()

# the window class like mentioned before (or like @ekhumoro wrote it)
class Window(QtWidgets.QWidget):
    def __init__(self):
        self._data_container = QtWidgets.QWidget()
        layout = QtWidgets.QVBoxLayout()
        self._data_container.setLayout(layout)

    def addParsedData(data):
        data_widget = ParsedDataWidget(data)

        layout = self._data_container.layout()
        layout.addWidget(data_widget)

So what do I have to do in order to get the addParsedData method to work?

Edit

I was trying some changings of the code. If I replace the ParsedDataWidget with a QLabel I get the following result:

If i close the window python crashes.

Solution

With some further research I found my problem: You should not use threads with PyQt, you should use SIGNALS instead (written here)

So I changed the code of the worker, I added another SIGNAL called finishedParsing which is emitted if the loading is completed. This SIGNAL holds the DataParser. The could would look like this:

class Worker(QtCore.QObject):
    finishedParsing = QtCore.pyqtSignal(DataParser)

    def run(self):
        self._stop = False
        for count, file in enumerate(self._files, 1):

            # parse the data in the DataParser and create an object
            # of the files data
            data = DataParser(file)

            # emit a signal to let the window know that this data is
            # ready to use
            self.finishedParsing.emit(data)

            self.loaded.emit(count, file)
            if self._stop:
                break
        self.finished.emit()

class Window(QtWidgets.QWidget):
    def showOpenDialog(self):
        if files and not self.thread.isRunning():
            # do the opening stuff like written before
            self.worker = Worker(files)

            #...

            self.worker.finishedParsing.connect(self.addParsedData)

This works now!

Answer 1

Below is an example that implements what you asked for. In real usage, the QThread.sleep line should be replaced with a function call that processes each file. This could either be defined as a method of the Worker class, or passed in as an argument to its __init__.

import sys, os
from PyQt5 import QtCore, QtWidgets

class Worker(QtCore.QObject):
    loaded = QtCore.pyqtSignal(int, str)
    finished = QtCore.pyqtSignal()

    def __init__(self, files):
        super().__init__()
        self._files = files

    def run(self):
        self._stop = False
        for count, file in enumerate(self._files, 1):
            QtCore.QThread.sleep(2) # process file...
            self.loaded.emit(count, file)
            if self._stop:
                break
        self.finished.emit()

    def stop(self):
        self._stop = True

class Window(QtWidgets.QWidget):
    def __init__(self):
        super(Window, self).__init__()
        self.button = QtWidgets.QPushButton('Choose Files')
        self.button.clicked.connect(self.showOpenDialog)
        layout = QtWidgets.QVBoxLayout(self)
        layout.addWidget(self.button)
        self.thread = QtCore.QThread()

    def showOpenDialog(self):
        files, filters = QtWidgets.QFileDialog.getOpenFileNames(
            self, 'Open file(s)', '',
            'Raw files (*.rw.dat);;Data files (*.dat)'
            ';;Text files (*.txt);;All files (*)',
            'All files (*)')
        if files and not self.thread.isRunning():
            self.worker = Worker(files)
            self.worker.moveToThread(self.thread)
            self.worker.finished.connect(self.thread.quit)
            self.thread.started.connect(self.worker.run)
            self.thread.finished.connect(self.worker.deleteLater)
            self.showProgress(
                'Loading file(s)...', len(files), self.worker.stop)
            self.worker.loaded.connect(self.updateProgress)
            self.thread.start()

    def updateProgress(self, count, file):
        if not self.progress.wasCanceled():
            self.progress.setLabelText(
                'Loaded: %s' % os.path.basename(file))
            self.progress.setValue(count)
        else:
            QtWidgets.QMessageBox.warning(
                self, 'Load Files', 'Loading Aborted!')

    def showProgress(self, text, length, handler):
        self.progress = QtWidgets.QProgressDialog(
            text, "Abort", 0, length, self)
        self.progress.setWindowModality(QtCore.Qt.WindowModal)
        self.progress.canceled.connect(
            handler, type=QtCore.Qt.DirectConnection)
        self.progress.forceShow()

if __name__ == '__main__':

    app = QtWidgets.QApplication(sys.argv)
    window = Window()
    window.setGeometry(600, 100, 100, 50)
    window.show()
    sys.exit(app.exec_())