python multiprocess.Pool show results in order in stdout

Question

In multiprocessing.Pool I am trying to show my prints in the the same order.

from multiprocessing import Pool
import time
def func(arg):
    time.sleep(0.001)
    print(arg, end=" ")

proc_pool = Pool(4)
proc_pool.map(func, range(30))

The output is: 0 1 8 9 10 11 14 15 6 7 16 17 4 5 12 13 18 19 2 3 or similar. It is not in the order 0 1 2 3 ...

I know that imap can give better ordering ... but it's still not exactly what I want. I could override the print function to save it into a variable and print them all at once - but I would prefer showing them as soon as they complete - not when all complete.

Answer 1

Given that a Pool distributes the elements of the list amongst the worker processes (which are scheduled by the operating system), there is no way that you can guarantee internal processing order with map.

The best you can probably do (with map) is change the input into a list of tuples (sequence_number, data), have the worker function return (sequence_number, result) and than sort by sequence_number.

If you want to start processing items as soon as they are finished, use imap or imap_unordered. Using imap will preserve the same order as the input iterable. If you use the same trick with the (sequence_number, result) tuple with imap, you can save the results in a list and print them once you have a sequence without gaps.