Expecting $end after first input

Question

I'm trying to use flex and bison to build a simple parser for a programming language.

When I was testing it from terminal(stdin), it always says that (the yyerror() is customized):

Error: syntax error, unexpected SOME_TOKEN, expecting $end, on line: 2

on the second input. In another word, it only works well on the first input.

I can only guess that the internal parse stack of bison didn't flush after the first statement.

Here is my bison code:

%{
#include <stdio.h>
#include <stdlib.h> 

#define YYDEBUG 1
%}

%token TOKEN_DECLARE TOKEN_SET TOKEN_STR TOKEN_SYMBOL EOL
%error-verbose

%%

statement: statement
    | declare_var EOL { puts("Reach declare EOL branch"); }
    | set_value EOL { puts("Reach set_value EOL branch"); }

;

declare_var: TOKEN_DECLARE TOKEN_SYMBOL { printf("Declare var\n"); }
        | TOKEN_DECLARE set_value
;

set_value : TOKEN_SYMBOL TOKEN_SET { printf("Set var"); }
        | set_value TOKEN_STR { printf(" to string\n"); }
        | set_value TOKEN_SYMBOL { printf(" to symbol\n"); }
;

%%

Here is my flex code

%%
[\t\n ] {/*Ignore*/}

var {
    return (TOKEN_DECLARE);
}

";" {
    //puts("Got EOL");
    return (EOL);
}

\"[a-zA-Z]+\" {
    return (TOKEN_STR);
} 
\'[a-zA-Z]+\' {
    return (TOKEN_STR);
}

[a-zA-Z]+ {
    return (TOKEN_SYMBOL);
}

"=" {
    return (TOKEN_SET);
}
%%

Thanks

Answer 1

Your yacc code only accepts a single statement -- after the EOL the only valid token is $end (end of file/input), so you get a syntax error on the second line. Also, the rule statement: statement is useless and can never be reduced (you should be getting an error message from yacc about it).

What you want is an explicit rule for matching multiple statements:

statements: statements statement
          | statement
;

which is the first (start) rule. Then your statement rule is just:

statement: declare_var EOL { puts("Reach declare EOL branch"); }
         | set_value EOL { puts("Reach set_value EOL branch"); }
;