Threading Performance Degradation when Handling Pixels

Question

When my Python script spawns one thread and runs the following piece of code, the run time is 0.8 seconds. When it spawns five threads and runs the code, the run time is ~5.0 seconds.

Obviously, I would like the code to run in ~0.8 seconds even with 5 or 15 threads. Why would this happen? I have used threading to improve the run time of other parts of my program, but for some reason here it bottlenecks. Also, I never spawned more than 60 threads total, so that should not impact performance.

# Open the image
imgx = Image.open(imgName)
imgx = imgx.convert("RGBA")
pix = imgx.load()                   


# Adjust dark pixels to black or white
for y in xrange(imgx.size[1]):
    for x in xrange(imgx.size[0]):

    # Get RGBA values for a pixel
    (first, second, third, alpha) = pix[x, y]

    # Ajust the RGBA values accordingly
    if (first > threshold) or (second > threshold) or (third > threshold):      
        first = 255
        second = 255
        third = 255
        alpha = 255
    else:                                       
        first = 0
        second = 0
        third = 0
        alpha = 255

    # Set new pixel values
    pix[x, y] = (first, second, third, alpha) 

Answer 1

The Python interpreter has a global lock (called the Global Interpreter Lock or GIL) that prevents pure Python code from running concurrently in more than one thread.

Iterating over the pixels individually in Python loops is quite inefficient anyway. You should make use of Numpy's vectorized functions that can act globally on the array. This will be much faster in a single thread, and has the additional advantage that Numpy releases the GIL during the array operations, so they can actually happen in parallel in multiple threads.

You probably don't even need to use threads for this application. Using multiple processes has much fewer subtleties than using threading.

Numpy code roughly equivalent to what you wrote would be

img = Image.open(imgName).convert("RGBA")
arr = numpy.array(img)

# Split the channels of the image by rotating the axes
r, g, b, a = arr.transpose(2, 0, 1)

# Create Boolean array: True means pixel is above threshold
bw = (r > threshold) | (g > threshold) | (b > threshold)

# Set R, G and B channel to the 255 times the B/W array
arr[:, :, :2] = 255 * bw[:, :, numpy.newaxis]

# Set alpha channel to 255
arr[:, :, 3] = 255

# Create new PIL image from array
new_img = Image.fromarray(arr)