Search code examples
antlrantlr4lexer

Can you conditionally change ANTLR lexer modes?


I'm working on a language where there is an outer grammar that defines objects and an inner grammar that defines code. The inner grammar is embedded in various places. The inner grammar starts with a BEGIN and ends with an END. This would be simple except the inner code grammar can have multiple BEGIN/END blocks within the code too. So my question is, can I reference count BEGIN/END pairs to only popmode when I have reached the last matching END pair? I'm guessing this must be possible, but I'm just not sure how best to do this in ANTLR. Below is an small example of the grammar:

procedure Foobar(somethingHappens: boolean)
  var
      MyRecord: Record "Some Table";
      MyDecimal: Decimal;
      otherBoolean: Boolean;
  begin
      if (somethingHappens) then
      BEGIN
        // do some stuff
      END
      // do other stuff
  end;

I have many words that are special tokens in the outer grammar, but can be used as normal identifiers in code within the inner grammar, hence my need for two modes.


Solution

  • Yes you can reference count BEGIN/END pairs by defining an appropriate variable in the @lexer::members {} block and manipulating it in actions attached to your lexer rules, conditionally calling pushMode() and popMode() and perhaps _modeStack.peek(). But you may not need to.

    On your BEGIN rule pushMode(BEGINEND_MODE).

    Code an additional rule to match BEGIN within BEGINEND_MODE which does the same, recursively entering BEGINEND_MODE.

    Code an END rule within BEGINEND_MODE that does a popMode().

    BEGIN:
       'BEGIN'
       ->pushMode(BEGINEND_MODE)
       ;
    
    [...]
    
    mode BEGINEND_MODE;
    
    BEGIN1:
       'BEGIN'
       ->pushMode(BEGINEND_MODE),type(BEGIN)
       ;
    
    END:
       'END'
       ->popMode()
       ;
    
    [...]