Search code examples
pythonmultithreadingmultiprocessingpool

Python - Multiprocessing Pool Class: Can the Thread Function contain arguments?


I want to know if it is possible to pass arguments into the threading pool funktion?

from multiprocessing.dummy import Pool as ThreadPool 

def mainFunc():
    myArray = ["A", "B", "C"]

    pool = ThreadPool(8) 
    pool.map(threadFunc, myArray)   # how is it possible to give >>threadFunc<< another argument >>text1<<
    pool.close()
    pool.join()

def threadFunc(text2):
    text1 = "Hello "                # this should be given as a argument  of the function
    print text1 + text2

mainFunc()

This is a simple example of what i want to do.. How can I give text1 as a second argument to the threadFunc function?

Solved:

I solved the problem with a simple global accessible variable ... but it was only a solution for the problem I had right now ... it is a greater idea to use the multiprocessing.Process instead ...

import sys
from multiprocessing.dummy import Pool as ThreadPool 

this = sys.modules[__name__] 

def mainFunc():
    myArray = ["A", "B", "C"]
    this.text1 = "Hello "

    pool = ThreadPool(8) 
    pool.map(threadFunc, myArray)   # how is it possible to give >>threadFunc<< another argument >>text1<<
    pool.close()
    pool.join()

def threadFunc(text2):
    print this.text1 + text2

mainFunc()

Solution

    1. The official python docs says

      A parallel equivalent of the map() built-in function (it supports only one iterable argument though). It blocks until the result is ready.

      Therefore it is impossible to add another argument, but you can always add text1 as the last element of your myArray and pop it out once you use it.

    2. You can also use multiprocessing.Process other than pool. In this case you can add another argument easily.

      def f(name):
          print 'hello', name
      
      if __name__ == '__main__':
          p = Process(target=f, args=('bob',))
          p.start()
          p.join()