Search code examples
pythongarbage-collectionmultiprocessingpython-multiprocessingpool

Python garbage collect in imap_unordered


In common case work with pool, python marks returned args as deleted and gc within some time will delete this object from memory e.g.

def main():
    for x in pool.imap_unordered(func, args):
        pass

if __name__ == "__main__":
    main()

But how will be work garbage collector in case launch pool without accumulate result?

def main():
    pool.imap_unordered(func, args)

if __name__ == "__main__":
    main()

If pool works constantly, have any chance to get overload memory?


Solution

  • Right way to iterate over generator without getting result is standard for _ in map(): pass, because Python will don't allocate new memory for each object in map generator and will set each object in one region.