Search code examples
pythonpython-3.xpython-3.8python-assignment-expression

With assignment expressions in Python 3.8, why do we need to use `as` in `with`?


Now that PEP 572 has been accepted, Python 3.8 is destined to have assignment expressions, so we can use an assignment expression in with, i.e.

with (f := open('file.txt')):
    for l in f:
        print(f)

instead of

with open('file.txt') as f:
    for l in f:
        print(f)

and it would work as before.

What use does the as keyword have with the with statement in Python 3.8? Isn't this against the Zen of Python: "There should be one -- and preferably only one -- obvious way to do it."?


When the feature was originally proposed, it wasn't clearly specified whether the assignment expression should be parenthesized in with and that

with f := open('file.txt'):
    for l in f:
        print(f)

could work. However, in Python 3.8a0,

with f := open('file.txt'):
    for l in f:
        print(f)

will cause

  File "<stdin>", line 1
    with f := open('file.txt'):
           ^
SyntaxError: invalid syntax

but the parenthesized expression works.


Solution

  • TL;DR: The behaviour is not the same for both constructs, even though there wouldn't be discernible differences between the 2 examples.

    You should almost never need := in a with statement, and sometimes it is very wrong. When in doubt, always use with ... as ... when you need the managed object within the with block.


    In with context_manager as managed, managed is bound to the return value of context_manager.__enter__(), whereas in with (managed := context_manager), managed is bound to the context_manager itself and the return value of the __enter__() method call is discarded. The behaviour is almost identical for open files, because their __enter__ method returns self.

    The first excerpt is roughly analogous to

    _mgr = (f := open('file.txt')) # `f` is assigned here, even if `__enter__` fails
    _mgr.__enter__()               # the return value is discarded
    
    exc = True
    try:
        try:
            BLOCK
        except:
            # The exceptional case is handled here
            exc = False
            if not _mgr.__exit__(*sys.exc_info()):
                raise
            # The exception is swallowed if exit() returns true
    finally:
        # The normal and non-local-goto cases are handled here
        if exc:
            _mgr.__exit__(None, None, None)
    

    whereas the as form would be

    _mgr = open('file.txt')   # 
    _value = _mgr.__enter__() # the return value is kept
    
    exc = True
    try:
        try:
            f = _value        # here f is bound to the return value of __enter__
                              # and therefore only when __enter__ succeeded
            BLOCK
        except:
            # The exceptional case is handled here
            exc = False
            if not _mgr.__exit__(*sys.exc_info()):
                raise
            # The exception is swallowed if exit() returns true
    finally:
        # The normal and non-local-goto cases are handled here
        if exc:
            _mgr.__exit__(None, None, None)
    

    i.e. with (f := open(...)) would set f to the return value of open, whereas with open(...) as f binds f to the return value of the implicit __enter__() method call.

    Now, in case of files and streams, file.__enter__() will return self if it succeeds, so the behaviour for these two approaches is almost the same - the only difference is in the event that __enter__ throws an exception.

    The fact that assignment expressions will often work instead of as is deceptive, because there are many classes where _mgr.__enter__() returns an object that is distinct from self. In that case an assignment expression works differently: the context manager is assigned, instead of the managed object. For example unittest.mock.patch is a context manager that will return the mock object. The documentation for it has the following example:

    >>> thing = object()
    >>> with patch('__main__.thing', new_callable=NonCallableMock) as mock_thing:
    ...     assert thing is mock_thing
    ...     thing()
    ...
    Traceback (most recent call last):
      ...
    TypeError: 'NonCallableMock' object is not callable
    

    Now, if it were to be written to use an assignment expression, the behaviour would be different:

    >>> thing = object()
    >>> with (mock_thing := patch('__main__.thing', new_callable=NonCallableMock)):
    ...     assert thing is mock_thing
    ...     thing()
    ...
    Traceback (most recent call last):
      ...
    AssertionError
    >>> thing
    <object object at 0x7f4aeb1ab1a0>
    >>> mock_thing
    <unittest.mock._patch object at 0x7f4ae910eeb8>
    

    mock_thing is now bound to the context manager instead of the new mock object.