overloading yylex() using flexlexer in c++

Question

I'm trying to write a simple parser in lex and doing it in c++. I started off by following the directions here

WIKILINK    \133{2}[^\135]+\135{2}
%option noyywrap
%option c++
%{
#include <fstream>
#include <iostream>
#include <string>
#include <vector>
#include <stdio.h>

#include <FlexLexer.h>

class xxFlexLexer : public yyFlexLexer
{
    public:
    virtual int yylex()
    {
        std::cout << "hello" << std::endl;
    }
private:
            vector<string> listOfLinks;
};

using namespace std;
%}

%%
{WIKILINK}    {
                            int size = YYLeng();
                            string link(YYText());
                            link.erase(YYLeng()-3, 2);
                            link.erase(0, 2);
                            listOfLinks.push_back(link);
                            }
%%

int main()
{
    ifstream in;
    ofstream out;

    in.open("in.txt");
    out.open("out.txt");

    yyFlexLexer lexer(&in, &out);
    lexer.yylex();
}

When i try to compile the program lex.yy.cc, I get the following errors -

In file included from tmp.cpp:12:0:
/usr/include/FlexLexer.h:112:7: error: redefinition of ‘class yyFlexLexer’
/usr/include/FlexLexer.h:112:7: error: previous definition of ‘class yyFlexLexer’
tmp.cpp: In member function ‘virtual int yyFlexLexer::yylex()’:
tmp.cpp:33:29: error: ‘listOfLinks’ was not declared in this scope

I don't understand this error. What is happening?

Answer 1

Looking on the web I found this FlexLexer.h include file https://www.w3.org/2005/05/22-SPARQL-MySQL/sparql/FlexLexer.h and it appears that the signature for yylex() has an argument FLEXFIX.

virtual int yylex(FLEXFIX) = 0;

// Call yylex with new input/output sources.
int yylex(FLEXFIX, istream* new_in, ostream* new_out = 0 )
    {
    switch_streams( new_in, new_out );
    return yylex(FLEXFIX2);
    }

However other versions of the file such as https://www.w3.org/2008/04/SPARQLfed/win/FlexLexer.h do not.

virtual int yylex() = 0;

// Call yylex with new input/output sources.
int yylex( FLEX_STD istream* new_in, FLEX_STD ostream* new_out = 0 )
    {
    switch_streams( new_in, new_out );
    return yylex();
    }

By the way your yylex() function is not returning an int value.

See also @Gir comment with a link to a bug report Bug 67277 - Error compiling libclassparser_la.all_cc.cc: "class yyFlexLexer" redefined.

The problem is that the FlexLexer.h header has a strange way of handling multiple inclusions of itself. tokenizer.l (tokenizer.cc) and ClassParser.cc both include this header indirectly, and in separate compilation they work fine; but when compiling concatenated source, a "#define yyFlexLexer yyFlexLexer" declaration (inserted by flex) carries through from tokenizer.cc to ClassParser.cc, and causes havoc on the second inclusion. (In the error message quoted above, the two FlexLexer.h lines are the second and first inclusions, respectively.)

The/A solution is to "#undef yyFlexLexer" at the end of tokenizer.l, and I will attach a patch that does this with an explanation why.

The comments in the FlexLexer.h include file mention using what is in the bug report for multiple instances and includes. Not sure why it would be needed in something this straightforward and simple though.

// This file defines FlexLexer, an abstract class which specifies the
// external interface provided to flex C++ lexer objects, and yyFlexLexer,
// which defines a particular lexer class.
//
// If you want to create multiple lexer classes, you use the -P flag
// to rename each yyFlexLexer to some other xxFlexLexer.  You then
// include <FlexLexer.h> in your other sources once per lexer class:
//
//  #undef yyFlexLexer
//  #define yyFlexLexer xxFlexLexer
//  #include <FlexLexer.h>
//
//  #undef yyFlexLexer
//  #define yyFlexLexer zzFlexLexer
//  #include <FlexLexer.h>
//  ...

Looking at the link you provided, I wonder if you need to remove the FlexLexer.h include since you have only the one parser.