Search code examples
pythonpython-3.xwith-statementpython-3.9contextmanager

Parenthesized context managers work in python 3.9 but not 3.8


So I have this simple example of a with statement. It works in Python 3.8 and 3.9:

class Foo:
    def __enter__(self, *args):
        print("enter")

    def __exit__(self, *args):
        print("exit")

with Foo() as f, Foo() as b:
    print("Foo")

Output (as expected):

enter
enter
Foo
exit
exit

But if I add parentheses like this it only works in Python 3.9:

class Foo:
    def __enter__(self, *args):
        print("enter")

    def __exit__(self, *args):
        print("exit")

with (Foo() as f, Foo() as b):
    print("Foo")

Output in 3.8:

  File "foo.py", line 8
    with (Foo() as f, Foo() as b):
                ^
SyntaxError: invalid syntax

I know, I could just remove the parentheses but I don't understand why it works in Python 3.9 in the first place. I could not find the relevant change log.


Solution

  • Parenthesized context managers are mentioned as a new feature in What’s New In Python 3.10. The changelog states:

    This new syntax uses the non LL(1) capacities of the new parser. Check PEP 617 for more details.

    But PEP 617 was already accepted in Python 3.9, as described in its changelog:

    Python 3.9 uses a new parser, based on PEG instead of LL(1). The new parser’s performance is roughly comparable to that of the old parser, but the PEG formalism is more flexible than LL(1) when it comes to designing new language features. We’ll start using this flexibility in Python 3.10 and later.

    It was even already part of the Python 3.9 grammar:

    with_stmt:
        | 'with' '(' ','.with_item+ ','? ')' ':' block 
        | 'with' ','.with_item+ ':' [TYPE_COMMENT] block 
        | ASYNC 'with' '(' ','.with_item+ ','? ')' ':' block 
        | ASYNC 'with' ','.with_item+ ':' [TYPE_COMMENT] block 
    with_item:
        | expression 'as' star_target &(',' | ')' | ':') 
        | expression
    

    My guess is that because the LL1 parser was officially removed in Python 3.10, only then it was considered a new feature, while still being supported in 3.9.