I need to run the same function 10 times that for reasons of data linked to a login, it needs to be inside another function:
from multiprocessing import Pool
def main():
def inside(a):
print(a)
Pool.map(inside, 'Ok' * 10)
if __name__ == '__main__':
main()
from multiprocessing import Pool
def main():
def inside(a):
print(a)
Pool.map(main.inside, 'Ok' * 10)
if __name__ == '__main__':
main()
In both attempts the result is this:
AttributeError: 'function' object has no attribute 'map'
How can I do this by keeping the function inside the other function?
Is there a way to do this?
AttributeError: 'function' object has no attribute 'map'
We need to instantiate Pool
from multiprocessing
and call map
method of that pool object.
You have to move inside
method to some class because Pool uses pickel to serialize and deserialize methods and if its inside some method then it cannot be imported by pickel
.
Pool needs to pickle (serialize) everything it sends to its worker-processes (IPC). Pickling actually only saves the name of a function and unpickling requires re-importing the function by name. For that to work, the function needs to be defined at the top-level, nested functions won't be importable by the child and already trying to pickle them raises an exception (more).
Please visit this link of SO.
from multiprocessing import Pool
class Wrap:
def inside(self, a):
print(a)
def main():
pool = Pool()
pool.map(Wrap().inside, 'Ok' * 10)
if __name__ == '__main__':
main()
If you don't want to wrap inside
method inside of a class
move the inside
method to global scope so it can be pickled
from multiprocessing import Pool
def inside(a):
print(a)
def main():
with Pool() as pool:
pool.map(inside, 'Ok'*10)
if __name__ == '__main__':
main()