Search code examples
antlrgrammarlexercontext-free-grammarparser-generator

ANTRL simple grammar and Identifier


I wrote this simple grammar for ANTLR

grammar ALang;

@members {

public static void main(String[] args) throws Exception {   
    ALangLexer lex = new ALangLexer(new ANTLRFileStream("antlr/ALang.al"));
    CommonTokenStream tokens = new CommonTokenStream(lex);
    ALangParser parser = new ALangParser(tokens);
    parser.prog();
} 

}

prog : 
    ID | PRINT
    ;

PRINT : 'print';
ID : ( 'a'..'z' | 'A'..'Z' )+;
WS : (' ' | '\t' | '\n' | '\r')+ { skip(); };

Using as input:

print

the only token found is a token of type ID. Isn't enough to put the PRINT token definition right before the ID definition?


Solution

  • ALang.g:21:1: The following token definitions can never be matched because prior tokens match the same input: PRINT

    Yes, that is enough. If you define PRINT after ID, ANTLR will produce an error:

    ALang.g:21:1: The following token definitions can never be matched because prior tokens match the same input: PRINT 
    

    I'm so sorry, i didn't want to use this production: PRINT : 'print '; but the production without the trailing space: PRINT : 'print'; The problem is that 'print' is derived from ID and not from PRINT

    No, that can't be the case.

    The following:

    grammar ALang;
    
    @members {
      public static void main(String[] args) throws Exception {  
        ALangLexer lex = new ALangLexer(new ANTLRStringStream("sprint print prints foo"));
        CommonTokenStream tokens = new CommonTokenStream(lex);
        ALangParser parser = new ALangParser(tokens);
        parser.prog();
      } 
    }
    
    prog
     : ( ID    {System.out.printf("ID    :: '\%s'\n", $ID.text);}
       | PRINT {System.out.printf("PRINT :: '\%s'\n", $PRINT.text);}
       )*
       EOF
     ;
    
    PRINT : 'print';
    ID    : ('a'..'z' | 'A'..'Z')+;
    WS    : (' ' | '\t' | '\n' | '\r')+ {skip();};
    

    will print:

    ID    :: 'sprint'
    PRINT :: 'print'
    ID    :: 'prints'
    ID    :: 'foo'
    

    As you see, the PRINT rule does match "print".