Flex Bison and Gnu Automake

Question

I wrote a small text-filter based on flex-lexer and bison. A boolean expression is applied to the specified string(logvar). Boolean expression consists of strings and boolean operations.

example:

*intput*:
logvar=aa bb cc dd ee
aa & bb
*result*:
true

*input*:
logvar=aa bb cc dd ee
aa & rr
*result*:
false

*input*:
logvar=aa bb cc dd ee
aa | rr
*result*:
true

Very simple.

Here are the sources lexer-filter.l:

%{
#include <stdlib.h>
void yyerror(char *);
#include "grammar-filter.tab.h"
%}

%%

"&" {
    printf("LEX: & parsed\n");
    return AND;
}

"|" {
    printf("LEX: | parsed\n");
    return OR;
}

"logvar" {
    printf("LEX: logvar parsed\n");
    return LOGVAR;
}

[a-z ]+ {
    yylval.sValue = strdup(yytext);
    printf("LEX: string parsed: %s\n", yylval.sValue);
    return STRING;
}

[()=\n] { return *yytext; }
[ \t] ;
, { return *yytext; }
. yyerror("invalid character");
%%
int yywrap(void) {
    return 1;
}

and grammar bison file:

%{
#include <string.h>
#include <stdio.h>
void yyerror(char *);
int yylex(void);
int sym[26];
char* log_line;
%}
%union {
    char *sValue;
    int iValue;
}

%token <iValue> BOOLEAN
%token <sValue> STRING
%token LOGVAR
%token AND OR
%left AND OR

%type<iValue> expr 

%%

prog :
prog logassigned '\n' statement '\n'
;

statement :
expr { printf("%d\n", $1); }
;
    
logassigned :
LOGVAR '=' STRING { log_line = $3; }
;

expr:
STRING { if (strstr(log_line, $1) != NULL) {
                printf("%s is substring of %s\n", $1, log_line);
                $$ = 1;
         } else {
                 printf("%s doesn't substring of %s\n", $1, log_line);
                 $$ = 0;
         }
       }
| expr OR expr { if ($1 == 1 || $3 == 1) { $$ = 1; } else { $$ = 0; } }
| expr AND expr { $$ = $1 * $3; }
| NOT expr { if ($2 == 1) { $$ = 0; } else { $$ = 1; } }
| '(' expr ')' { $$ = $2; }
;

%%
#include "lex.yy.c"
void yyerror(char *s) {
        fprintf(stderr, "%s\n", s);
}

int main(void) {
//        yy_scan_string(some_string_buffer);
        yyparse();
//        yylex_destroy();

    return 0;
}

the final binary file is build by a simple make file:

all:
    flex lexer-filter.l
    bison grammar-filter.y -d
    gcc grammar-filter.tab.c -ll

I have a library based on gnu autotools (automake, autoconf...). Is it possible to add this parser to library and use this three construction yy_scan_string(some_string_buffer); yyparse(); yylex_destroy(); in my library? And how can I run flex and bison from automake scripts to generate files(lex.yy.c grammar-filter.tab.c grammar-filter.tab.h)

regards, max)

regards, max

Answer 1

Autotools has built-in support for flex and bison, although it's a bit quirky (or "idiosyncratic" as the linked documentation says).

At a minimum, you need to do two things:

1. Add AC_PROG_LEX and AC_PROG_YACC to configure.ac.

2. Add a definition of BUILT_SOURCES which lists the generated header file to Makefile.am.

See this answer for an example.

You'll also want to clean your project up a bit. Autotools assumes that the parser and scanner will be built separately, so you'll need to separate them. #includeing the scanner in the parser may seem convenient but it's not really good project organisation. (In other words, this requirement is not one of Autotools' quirks.) You'll need to #include headers as appropriate: the bison-generated header file needs to be #includedd in the scanner source, and the flex header file (which you can request with a flex option) needs to be #included in the source code which calls the flex functions.

Instead of using the flex-generated header directly, you might want to write your own little wrapper into the scanner itself, and write your own header file for the wrapper functions. I usually do that to avoid a tight dependency on flex.

You can improve compilability of the scanner by adding a %option declaration to the scanner, at a minimum:

%option noinput nounput noyywrap

The first two options avoid compiler warnings about unused functions and the last one eliminates the need for yywrap, and consequently the need for -ll (or -lfl). I usually add nodefault as well, so that flex will produce an error if some input would fall back to the default rule, which usually indicates a bug in the scanner specification.

On the whole, it is usually best to include the generated files in the distribution, rather than insist that the correct versions of flex and bison be present. Autotools can be used in this fashion as well.