PyQt and QThreads: separate threads still freezing main GUI thread?

Question

I am trying to do some work in a separate thread, in Python. I'm using PyQt so my threads are QThreads.

The work itself involves loading large images with PIL and converting the image to a QImage.

Here is an example of some code that loads an image over and over within a worker thread:

class testThread(QtCore.QThread):
    def run(self):
        while True:
            im = Image.open('c:\zheadline.tif')
            try:
                imgData = ImageQt.ImageQt(im)
                print (imgData.width())
            except:
                print ("Image load error")

The image is quite large (16k), so it can take a couple of seconds to actually load it. However, during this time, the GUI becomes very unresponsive, often freezing for a few hundred milliseconds.

Why is this? My machine has 16 cores so there's no reason why the work from 1 thread would cause a slowdown in all of the others, right?

Basically I want to be able to load these large images in the background without interrupting the user experience at all. Does anyone have any ideas how I can push the main work off to another thread without disturbing the main GUI thread at all?

Some other info: I'm starting the QThreads with 'thread.start()'. Also, the GUI doesn't completely freeze during the load time...it just becomes extremely laggy and slow, freezing intermittently.

Answer 1

The python interpreter can only run in one thread at a time (read about the Global Interpreter Lock), so probably one or both of Image.open and ImageQt.ImageQt is doing a lot of work in python without releasing the GIL.

A possible solution is to load images in a separate process and transfer the data via shared memory. Pyqtgraph supports remote widget rendering this way: https://github.com/pyqtgraph/pyqtgraph/blob/master/pyqtgraph/widgets/RemoteGraphicsView.py. See the 'remoteSceneChanged' method for how the QImage was reconstructed from shared memory.