Search code examples
pythongeneratorcontextmanager

Is a context manager exited if a generator is garbage collected?


Suppose we have a simple generator like this:

def generator():
    with open('example.txt', 'r') as f:
        yield f.readline()
        yield f.readline()

and use it like this to get a single element/line:

line = next(generator())

Is the context manager closed once the temporary generator object becomes garbage collected?


Solution

  • A with statement has the same semantics as a try statement, with some additional code execute aside from the body of the with statement itself. From https://docs.python.org/3/reference/compound_stmts.html#the-with-statement"

    The following code:

    with EXPRESSION as TARGET:
        SUITE
    

    is semantically equivalent to:

    manager = (EXPRESSION)
    enter = type(manager).__enter__
    exit = type(manager).__exit__
    value = enter(manager)
    hit_except = False
    
    try:
        TARGET = value
        SUITE
    except:
        hit_except = True
        if not exit(manager, *sys.exc_info()):
            raise
    finally:
        if not hit_except:
            exit(manager, None, None, None)
    

    So our yield expressions are inside the try statement implied by the with statement.

    From https://docs.python.org/3/reference/expressions.html#yieldexpr

    Yield expressions are allowed anywhere in a try construct. If the generator is not resumed before it is finalized (by reaching a zero reference count or by being garbage collected), the generator-iterator’s close() method will be called, allowing any pending finally clauses to execute.

    The finally clause implied by the with statement explicitly calls the __exit__ method of the context manager, which closes the file.