Search code examples
pythonmultiprocessingthreadpoolpool

Passing list to pool.map() function in Python returns each character for each item in the list


This Python 2.7 script running on Windows is not outputting as I would expect it to.

from multiprocessing import Pool
from multiprocessing.dummy import Pool as ThreadPool

def function1():
    function1List = ["11" ,'22' , '33']
    return function1List

def function2(passedList):
    for item in passedList:
        print item

list = function1()
#print list    used to verify list

if __name__ == '__main__':
    pool = ThreadPool()
    pool.map(function2, list)
    pool.close() 
    pool.join()

What I expect

11
22
33
44

What I am actually getting

1
1
2
2
3
3
4
4

What am I missing?


Solution

  • map is passing one item of each list to function2.

    And then, inside function2, you're iterating over that item, getting each character of the string.

    It's exactly the same as if you'd written this:

    values = ["11" ,'22' , '33']
    for value in values:
        for item in value:
            print item
    

    So, how do you fix it? Well, it depends what you want.

    If you want each task to get a single list item, as it currently does, you just need to change function2 to expect that:

    def function2(item):
        print item
    

    If you want each task to get a list, you have to change the thing you call map on into a list of lists. I'm not sure what lists you want, but you could pass four single-element lists:

    values = [[value] for value in values]
    

    … or one four-element list:

    values = [values]
    

    … or two two-element interleaved lists:

    values = [values[::2], values[1::2]]
    

    … or anything else you want.