why does pool imap_unordered not work where map does in this case?

Question

It is my understanding that imap_unordered and map should be interchangeable, if you don't care about the order of outputs. I've had this happen a few times now though, imap_unordered just doesn't work at all. I normally use with partial but here have switched to a tuple . Any help appreciated. Python 3.11.2

#!/usr/bin/env python3

from multiprocessing import Pool
from functools import partial

def main ():
    query_seqs = list(range(22))
    print(len(query_seqs))
    with Pool(processes=7) as pool:
        tmp = [('args2', 'blast_type2', query) for query in query_seqs]
        pool.map(pool_tasks, tmp)
    print(len(query_seqs))
    with Pool(processes=7) as pool:
        tmp = [('args1', 'blast_type1', query) for query in query_seqs]
        pool.imap_unordered(pool_tasks, tmp)
    print(len(query_seqs))
def pool_tasks(tup):
    args, blast_type, query = tup
    print('args, blast_type, query', args, blast_type, query)

if __name__ == '__main__':
    main()

Answer 1

I'm guessing your problem is that you don't see any output of the form

args, blast_type, query args1 blast_type1 0

which causes you to believe that the unordered version "just doesn't work at all". Right?

But it does. You're letting that part of the code destroy the pool before any of the unordered tasks can run. Change this:

        pool.imap_unordered(pool_tasks, tmp)

to, for example, this:

        for x in pool.imap_unordered(pool_tasks, tmp):
            pass

Then the code won't fall off the end of the with Pool block before the unordered work completes.

In contrast, the ordered version constructs a list of the results, which is created regardless of whether you bind that list to a name. It stays in its with Pool block until the work is complete.

But the unordered version doesn't construct a list. It returns an iterator immediately and moves on.