Search code examples
parsingecmascript-6context-free-grammarcompiler-construction

How does `yield` and `await` production parameters expand into `*` and `async` before identifiers


Can anyone explain how the following production expands into * and async keywords:

BindingIdentifier[Yield, Await]:
   Identifier
   [~Yield]yield
   [~Await]await

Based on the TypeScript parsing code I see that it checks for * and async keywords, so I assume here that BindingIdentifier_Yield matches *identifier and BindingIdentifier_Await matches async identifier but I can't seem to trace that expansion using the above grammar.

I know I can expand the identifiers [Yield, Await] according to the spec:

A production may be parameterized by a subscripted annotation of the form “[parameters]”... A parameterized production is shorthand for a set of productions defining all combinations of the parameter names, preceded by an underscore, appended to the parameterized nonterminal symbol

So the above is expanded into:

BindingIdentifier:
   Identifier
   [~Yield]yield
   [~Await]await

BindingIdentifier_Yield:
   Identifier
   [~Yield]yield
   [~Await]await

BindingIdentifier_Await:
   Identifier
   [~Yield]yield
   [~Await]await

But how then BindingIdentifier_Yield and BindingIdentifier_Await is expanded into * and async? I suspect that the explanation is here:

[~Yield]yield
[~Await]await

but I'm not sure. Any help is welcome!


Solution

  • The check for the * token in Typescript exists to handle both

    14.1:

    FunctionExpression[Yield, Await, Default]:
        function BindingIdentifier[?Yield, ?Await]opt ( FormalParameters[~Yield, ~Await] ) { FunctionBody[~Yield, ~Await] }
    

    vs

    14.4:

    GeneratorExpression[Yield, Await, Default]:
        function* BindingIdentifier[?Yield, ?Await]opt ( FormalParameters[+Yield, ~Await] ) { GeneratorBody }
    

    since both function and generator expression can be expanded from PrimaryExpression:

    PrimaryExpression:
        this
        Literal
        ArrayLiteral
        ObjectLiteral
        FunctionExpression       <---------
        ClassExpression 
        GeneratorExpression      <---------
        AsyncFunctionExpression
        RegularExpressionLiteral
        TemplateLiteral
    

    These checks also overlap for function declarations, and method syntax.

    The production

    BindingIdentifier [Yield, Await]:
        Identifier
        [~Yield] yield
        [~Await] await
    

    expands to

    BindingIdentifier:
        Identifier
        yield
        await
    
    BindingIdentifier_Yield:
        Identifier
        await
    
    BindingIdentifier_Await:
        Identifier
        yield
    
    BindingIdentifier_Yield_Await:
        Identifier
    

    So yield and await identifiers are not allowed in cases where +Yield and/or +Await has been used in the grammar. You can see in the examples above, they use

    FormalParameters[~Yield, ~Await]
    FunctionBody[~Yield, ~Await]
    

    whereas the generator uses

    FormalParameters[+Yield, ~Await]
    FunctionBody[+Yield, ~Await]
    

    Since the generator says +Yield instead of ~Yield, it means that

    function foo(){ var yield; } // works
    function* foo(){ var yield; } // not allowed