Search code examples
pythonparallel-processingconcurrent.futures

Why ProcessPoolExecutor working serially?


from concurrent.futures import ProcessPoolExecutor
import os
import time

def parInnerLoop(item):
    print(f'Processing {os.getpid()} started on {item}')
    time.sleep(3)
    print(f'Processing {os.getpid()} done on {item}')

def main():
    executor = ProcessPoolExecutor(max_workers=4)

    for itemNo in range(10):
        executor.submit(parInnerLoop(itemNo))

if __name__ == '__main__':
    main()

What I'm trying to achieve is parallel for loop, similar to MatLab, e.g.:

parfor itemNo = 0:9
    parInnerLoop(itemNo);
end

What I'm getting: all os.getpid are same, and execution occurs serially. Any help?

Windows, VSCodium/VSCode, Python 3.7.3


Solution

  • As mentioned by @Klaus, need to change from executor.submit(parInnerLoop(itemNo)) to executor.submit(parInnerLoop, itemNo).