In Python, how do I pass local variable values to multiprocessing library's Pool?

Question

I struggle with using the multiprocessing library in Python, where I want to use Pool.

Sample code (actual code is far more complex, but would not be helpful in demonstrating the problem):

def Multicore_Patterns_List_processor(Variable1, Variable2, Variable3):
    print(f"Variable1 = {Variable1}, Variable2 = {Variable2}, Variable3 = {Variable3}, current pattern is {Pattern}")

Variable1 = 1
Variable2 = 2
Variable3 = 3
MultiCores = 3
Paterns_list = list(range(1, 21))
with Pool(processes=MultiCores) as pool:
   pool.imap(Multicore_Patterns_List_processor(Variable1=Variable1, Variable2=Variable2, Variable3=Variable3), Paterns_list)

What should the code do:

1. The code should hand over all variables and the pattern selected (but I don't know how to do it)
2. There is no need to change the values of Variable1, etc. as these are in the example used to demonstrate a need to transfer some variables to each call
3. Why Pool? Because, in the actual code, I also use variables for different processes that select the number of processors dynamically

Thank you for any help

Jan Vaško

Note: The expectation is that each function call will work with the variables passed and also the iterator used for multiprocessing.

Answer 1

There are several ways to achieve your objective. If you want to use imap() then you could do it like this:

from multiprocessing import Pool

def Multicore_Patterns_List_processor(args):
    Variable1, Variable2, Variable3, Pattern = args
    print(f"{Variable1=}, {Variable2=}, {Variable3=}, current pattern is {Pattern}")

def genargs(Variable1, Variable2, Variable3):
    for p in range(10):
        yield Variable1, Variable2, Variable3, p

if __name__ == "__main__":
    Variable1 = 1
    Variable2 = 2
    Variable3 = 3
    MultiCores = 3
    with Pool(processes=MultiCores) as pool:
        pool.imap(Multicore_Patterns_List_processor, genargs(Variable1, Variable2, Variable3))
        pool.close()
        pool.join()

Output:

Variable1=1, Variable2=2, Variable3=3, current pattern is 0
Variable1=1, Variable2=2, Variable3=3, current pattern is 1
Variable1=1, Variable2=2, Variable3=3, current pattern is 2
Variable1=1, Variable2=2, Variable3=3, current pattern is 3
Variable1=1, Variable2=2, Variable3=3, current pattern is 4
Variable1=1, Variable2=2, Variable3=3, current pattern is 5
Variable1=1, Variable2=2, Variable3=3, current pattern is 6
Variable1=1, Variable2=2, Variable3=3, current pattern is 7
Variable1=1, Variable2=2, Variable3=3, current pattern is 8
Variable1=1, Variable2=2, Variable3=3, current pattern is 9