Search code examples
pythongeneratorcontextmanager

Yielding from within with statement and __exit__ method of context manager

Consider following Python 2.x code snippet.

from __future__ import print_function


class myfile(file):
    def __exit__(self, *excinfo):
        print("__exit__ called")
        super(myfile, self).__exit__(*excinfo)


def my_generator(file_name):
    with myfile(file_name) as fh:
        for line in fh:
            yield line.strip()


gen = my_generator('file.txt')
print(next(gen))
print("Before del")
del gen
print("After del")

Output of this script (given file.txt has more than one line) is:

Line 1 from file
Before del
__exit__ called
After del

I'm interested about __exit__ call specifically.

What triggers execution of his method? For what we know, code never left with statement (it "stopped" after yield statement and never continued). Is it guaranteed that __exit__ will be called when reference count of generator drops to 0?

Solution

  • On reclamation of a generator object, Python calls its close method, raising a GeneratorExit exception at the point of its last yield if it wasn't already finished executing. As this GeneratorExit propagates, it triggers the __exit__ method of the context manager you used.

    This was introduced in Python 2.5, in the same PEP as send and yield expressions. Before then, you couldn't yield inside a try with a finally, and if with statements had existed pre-2.5, you wouldn't have been able to yield inside one either.