When QListWidget loads a large number of image, it will become very slow. Is there a way to load by QScrollBar? Or is there a better way?

I want to get a list of image thumbnails, When QListWidget loads a large number of image, it will become very slow.More than 200 pictures will take 5s to load. One-time loading seems to be a stupid way:

from PyQt5 import QtWidgets
from PyQt5 import QtGui
from PyQt5 import QtCore
from PyQt5 import Qt
import os

class MyQListWidgetItem(QtWidgets.QListWidgetItem):
    '''icon item'''
    def __init__(self, path, parent=None):
        self.icon = QtGui.QIcon(path)
        super(MyQListWidgetItem, self).__init__(self.icon, '', parent)

class MyQListWidget(QtWidgets.QListWidget):
    def __init__(self):
        super(MyQListWidget, self).__init__()
        path = './imgpath'
        self.setFlow(QtWidgets.QListView.LeftToRight)
        self.setIconSize(QtCore.QSize(180, 160))
        self.setResizeMode(Qt.QListWidget.Adjust)

        #add icon
        for fp in os.listdir(path):
            self.addItem(MyQListWidgetItem(os.path.join(path, fp), self))


if __name__ == "__main__":
    import sys
    app = QtWidgets.QApplication(sys.argv)
    w = MyQListWidget()
    w.show()
    sys.exit(app.exec_())

Solution

There are various possible approaches for this.

The important aspect is to delay the loading of images until it's actually required.

In the following example I used two custom roles to simplify the process: PathRole contains the full path to the image, ImageRequestedRole is a "flag" that tells if the image has been already loaded (or queued for loading).

The priority obviously goes to the images that are currently visible in the viewport, and we need to ensure that whenever the visible area changes the images are loaded as soon as possible.

To achieve that, I connected the scroll bar valueChanged and rangeChanged signals (the latter is mostly required on startup) to a function that checks the range of visible indexes and verifies whether they contain a path and if they have not been loaded nor queued yet. This will also queue loading of images whenever the window is enlarged to a bigger size which would show items previously hidden.

Once the function above finds that some images require loading, they are queued, and a timer is started (if not already active): using a timer ensures that loading is progressive and doesn't block the whole UI until all requested images are processed.

Some important aspects:

images are not stored as their source (otherwise you'll easily end up all resources), but scaled down.
a "lazy loader" ensures that images that are not currently shown are lazily loaded as soon as the current queue is completed; note that if you plan to browse through huge amount of images that are also very big, this is not suggested.
since the images are not loaded instantly, the items don't have a correct size by default: setting the icon size is not sufficient, as that size not considered until the item actually has a "decoration"; to work around that, a delegate is used, which implements the sizeHint method and sets a decoration size even if the image is not yet loaded: this ensures that the view already reserves enough space for each item without continuously computing positions relative to every other item.
setting the "loaded" flag requires writing data on the model, which by default causes the view to compute again sizes; to avoid that, a temporary signal blocker is used, so that the model is updated without letting it know to the view.
for performance reasons, you cannot have different widths for each image depending on the image aspect ratio.

PathRole = QtCore.Qt.UserRole + 1
ImageRequestedRole = PathRole + 1


class ImageDelegate(QtWidgets.QStyledItemDelegate):
    def initStyleOption(self, opt, index):
        super().initStyleOption(opt, index)
        if index.data(PathRole):
            opt.features |= opt.HasDecoration
            opt.decorationSize = QtCore.QSize(180, 160)


class MyQListWidget(QtWidgets.QListWidget):
    def __init__(self):
        super(MyQListWidget, self).__init__()
        path = './imgpath'
        self.setFlow(QtWidgets.QListView.LeftToRight)
        self.setIconSize(QtCore.QSize(180, 160))
        self.setResizeMode(Qt.QListWidget.Adjust)

        for fp in os.listdir(path):
            imagePath = os.path.join(path, fp)
            item = QtWidgets.QListWidgetItem()
            if os.path.isfile(imagePath):
                item.setData(PathRole, imagePath)
            self.addItem(item)

        self.imageDelegate = ImageDelegate(self)
        self.setItemDelegate(self.imageDelegate)
        self.imageQueue = []
        self.loadTimer = QtCore.QTimer(
            interval=25, timeout=self.loadImage, singleShot=True)
        self.lazyTimer = QtCore.QTimer(
            interval=100, timeout=self.lazyLoadImage, singleShot=True)
        self.lazyIndex = 0

        self.horizontalScrollBar().valueChanged.connect(self.checkVisible)
        self.horizontalScrollBar().rangeChanged.connect(self.checkVisible)

    def checkVisible(self):
        start = self.indexAt(QtCore.QPoint()).row()
        end = self.indexAt(self.viewport().rect().bottomRight()).row()
        if end < 0:
            end = start
        model = self.model()
        for row in range(start, end + 1):
            index = model.index(row, 0)
            if not index.data(ImageRequestedRole) and index.data(PathRole):
                with QtCore.QSignalBlocker(model):
                    model.setData(index, True, ImageRequestedRole)
                self.imageQueue.append(index)
        if self.imageQueue and not self.loadTimer.isActive():
            self.loadTimer.start()

    def requestImage(self, index):
        with QtCore.QSignalBlocker(self.model()):
            self.model().setData(index, True, ImageRequestedRole)
        self.imageQueue.append(index)
        if not self.loadTimer.isActive():
            self.loadTimer.start()

    def loadImage(self):
        if not self.imageQueue:
            return
        index = self.imageQueue.pop()
        image = QtGui.QPixmap(index.data(PathRole))
        if not image.isNull():
            self.model().setData(
                index, 
                image.scaled(self.iconSize(), QtCore.Qt.KeepAspectRatio), 
                QtCore.Qt.DecorationRole
            )
        if self.imageQueue:
            self.loadTimer.start()
        else:
            self.lazyTimer.start()

    def lazyLoadImage(self):
        self.lazyIndex += 1
        if self.lazyIndex >= self.count():
            return
        index = self.model().index(self.lazyIndex, 0)
        if not index.data(ImageRequestedRole) and index.data(PathRole):
            with QtCore.QSignalBlocker(self.model()):
                self.model().setData(index, True, ImageRequestedRole)
            image = QtGui.QPixmap(index.data(PathRole))
            if not image.isNull():
                self.model().setData(
                    index, 
                    image.scaled(self.iconSize(), QtCore.Qt.KeepAspectRatio), 
                    QtCore.Qt.DecorationRole
                )
        else:
            self.lazyLoadImage()
            return
        if not self.imageQueue:
            self.lazyTimer.start()

Finally, consider that this is a very basic and simple implementation for learning purposes:

an image viewer should not store all images in memory (not even as thumbnails as in my example): consider that images are stored as raster ("bitmap"), so even a thumbnail could occupy much more memory than the original compressed image;
a cache could be used in a temporary path in case a maximum amount of thumbnail is reached;
image loading should happen in a separate thread, possibly displaying a placeholder until the process is complete;
appropriate checks should be done to ensure that a file is actually an image, and/or if the image is corrupted;
unless you plan to show something else beside the thumbnail (file name, stats, etc), you should probably consider implementing the paint function of the delegate, otherwise some margin will always be shown on the right of the image;