Search code examples
pythonmultiprocessingkeyword-argument

Python Multiprocessing - How to pass kwargs to function?


How do I pass a dictionary to a function with Python's Multiprocessing? The Documentation: https://docs.python.org/3.4/library/multiprocessing.html#reference says to pass a dictionary, but I keep getting

TypeError: fp() got multiple values for argument 'what'

Here's the code:

from multiprocessing import Process, Manager
   
def fp(name, numList=None, what='no'):
        print ('hello %s %s'% (name, what))
        numList.append(name+'44')
 
if __name__ == '__main__':

    manager = Manager()
 
    numList = manager.list()
    for i in range(10):
        keywords = {'what':'yes'}
        p = Process(target=fp, args=('bob'+str(i)), kwargs={'what':'yes'})
        p.start()
        print("Start done")
        p.join()
        print("Join done")
    print (numList)

Solution

  • When I ran your code, I got a different error:

    TypeError: fp() takes at most 3 arguments (5 given)
    

    I debugged by printing args and kwargs and changing the method to fp(*args, **kwargs) and noticed that "bob_" was being passed in as an array of letters. It seems that the parentheses used for args were operational and not actually giving you a tuple. Changing it to the list, then also passing in numList as a keyword argument, made the code work for me.

    from multiprocessing import Process, Manager
    
    def fp(name, numList=None, what='no'):
        print ('hello %s %s' % (name, what))
        numList.append(name+'44')
    
    if __name__ == '__main__':
    
        manager = Manager()
    
        numList = manager.list()
        for i in range(10):
            keywords = {'what': 'yes', 'numList': numList}
            p = Process(target=fp, args=['bob'+str(i)], kwargs=keywords)
            p.start()
            print("Start done")
            p.join()
            print("Join done")
        print (numList)