MainWindow do not process events and user actions concurrently

Question

I have a PyQt5 MainWindow that I made using Qt Designer.

This window is supposed to crawl a site and list crawled links in a TreeView once they're found.

I am able to do that by creating a model, QStandardItemModel each time a new link is crawled, adding all links to the model and then set the model to the tree with TreeView.setModel(model), and then I'd call QtWidgets.qApp.processEvents()so the window is updated. While the software crawles the site, the window does not respond to user interactions, until QtWidgets.qApp.processEvents() is executed.

How can I update my TreeView with the latest crawled link and still have my window intractable during the crawling process?

Here's my code

def start(self):
    ## start crawling
    self.populate_tree(crawled_links)

def populate_tree(self, links):
    data = []
    for index in links:
        item = (links[index]['from'], [])
        for link in links[index]['url']:
            item[1].append((link, []))
        data.append(item)
    model = QtGui.QStandardItemModel()
    self.__add_items(model, data)
    self.treeView.setModel(model)
    self.treeView.expandAll()
    self.treeView.scrollToBottom()
    self.treeView.setHeaderHidden(True)
    QtWidgets.qApp.processEvents()

def __add_items(self, model, data):
    for text, children in data:
        item = QtGui.QStandardItem(text)
        model.appendRow(item)
        if children:
            self.__add_items(item, children)

if it's any help, crawled_links list looks like this:

crawled_links = {
0:{
    'url': {
        'one.html',
        'three.html'
        },
    'from':
        'site1.com'
    },
1:{
    'url': {
        'two.html'
        },
    'from':
        'site1.com'
    }
}

Answer 1

Using processEvents() is a bad practice, they should only be used for certain special tasks and as you see does not solve the problem.

The solution is to perform the task on a thread and send it through signals to the thread of the GUI.

---

import threading

from PyQt5 import QtCore, QtWidgets

class Helper(QtCore.QObject):
     resultChanged = QtCore.pyqtSignal(dict)

     def start_crawling(self):
         crawled_links = {}
         # processing
         self.resultChanged.emit(crawled_links)


class MainWindow(QtWidgets.QMainWindow, Ui_MainWindow):
     def __init__(self, parent=None)
         QtWidgets.QMainWindow.__init__(self, parent)
         self.setupUi(self)
         self.helper = Helper()
         self.helper.resultChanged.connect(self.populate_tree)

     def start(self):
         # start crawling
         threading.Thread(target=self.helper.start_crawling, daemon=True).start()


     def populate_tree(self, crawled_links)
         ...