problem with using threads instead of processes in python

Question

so i found a tutorial on multiprocessing and on threading on youtube by Corey Schafer. in the multiprocessing video we have 15 photos that we want to add blur to them and add them to a different folder called 'processed' in the existing one.

import time
import concurrent.futures
from PIL import Image, ImageFilter

img_names = [
'photo-1516117172878-fd2c41f4a759.jpg',
'photo-1532009324734-20a7a5813719.jpg',
'photo-1524429656589-6633a470097c.jpg',
'photo-1530224264768-7ff8c1789d79.jpg',
'photo-1564135624576-c5c88640f235.jpg',
'photo-1541698444083-023c97d3f4b6.jpg',
'photo-1522364723953-452d3431c267.jpg',
'photo-1513938709626-033611b8cc03.jpg',
'photo-1507143550189-fed454f93097.jpg',
'photo-1493976040374-85c8e12f0c0e.jpg',
'photo-1504198453319-5ce911bafcde.jpg',
'photo-1530122037265-a5f1f91d3b99.jpg',
'photo-1516972810927-80185027ca84.jpg',
'photo-1550439062-609e1531270e.jpg',
'photo-1549692520-acc6669e2f0c.jpg'
]

def process_image(img_name):

    size = (1200,1200)
    img = Image.open(img_name)

    img = img.filter(ImageFilter.GaussianBlur(15))
    img.thumbnail(size)
    img.save(f'processed/{img_name}')
    print(f"{img_name} was processed")

def main():

    t1 = time.perf_counter()

    with concurrent.futures.ProcessPoolExecutor() as executor:
        executor.map(process_image,img_names)

    t2 = time.perf_counter()

    print(f"finished in {t2-t1} seconds")

if __name__ == "__main__":
    main()

the problem is that when i try to use

with concurrent.futures.ThreadPoolExecutor() as executor: 

instead of what i have now to see how long it takes with threading then only half of the pictures get processed and added to the new folder, everything works just fine with with concurrent.futures.ProcessPoolExecutor() as executor: but not with threading. can someone explain why?

btw all the photos are saved in the same folder i have open now with the python file.

Answer 1

i managed to solve it by adding a max_workers arg in the pool, again, does anyone know why this was needed? i thought that if i don't give a value to it then the pool will take care of that and use as many as possible for my system.

with concurrent.futures.ThreadPoolExecutor(max_workers=8) as executor:

it seems to brake if i try to use a too large amount of threads or processes than my CPU can handle, im running an i7 4790K (4 cores, 8 threads) CPU so whenever i tried to use more than 8 threads or processes the program stopped processing the next photos, after running about half of them