Search code examples
elispbnfsmie

Using Emac's SMIE to parse language with optional ; for statement termination

I am using SMIE to parse a language that doesn't always require ; to terminate a statement. If the end of a line is outside of a brace construct ({},(),[]) and the last non-comment token was not an operator, then the \n acts as a statement terminator. Otherwise, if the end of the line is within a brace construct or the last token was an operator, then the \n acts as a continuation.

For example,

variable := 172 + 92; 

variable := 172 + 92

variable :=
    172 + 92;

variable :=
    172 + 92

variable := (172 +
    92)

are all valid statements. But,

variable 
    := 172 + 92

is not.

How can I encode this is the BNF grammar for SMIE (or any BNF for starters)? Or, is that not possible?

I understand how I might put this into the lexer and add ; tokens as appropriate, but I would rather put it into the grammar if possible.

Solution

  • No, you can't encode it in the BNF (because SMIE only accepts very weak BNFs which can't express that). Look at how I did it for Octave mode: the tokenizer is made to return ";" when it encounters a \n that is outside of a brace/bracket/paren (which you can check with (zerop (car (syntax-ppss)))).