Search code examples
pythonmultiprocessingpool

Python function parameters for Pool.map

 from multiprocessing import Pool
 from urllib.request import urlretrieve     
 tmp= Pool(4).map(urlretrieve, urls)

How can i specifiy the filename for urlretrieve in that case?

Solution

  • If you want to specify multiple arguments and you don't want to have to unpack them in the function urlretrieve (which you don't have access too) then perhaps you can try using the starmap. Example, here we are passing in both the url and the filename:

    from multiprocessing import Pool
from urllib.request import urlretrieve
urls = ["http://www.google.com", "http://www.yahoo.com"]
filenames = ["google.html", "yahoo.html"]
results = Pool(2).starmap(urlretrieve, zip(urls, filenames))