Hello I am trying to use Flex with C++. After reading the Flex Manual I started working on a small example to build my project off.
I came up with this small example initially.
#include <string.h>
#include <iostream>
#include <sstream>
#include <string>
#include <streambuf>
#include <fstream>
int main() {
std::string stringvalues = "TOKEN TEST 123";
std::istringstream inputStringStream(stringvalues);
std::istream& inputStream = inputStringStream;
std::ofstream outputStream("test.txt");
yyFlexLexer myTest(inputStream, outputStream);
}
Inside a build folder when I do cmake .. then hit make I get the following error:
alfredo@alfredo-ThinkPad-T440p:~/repos/FlexCpp/build$ make
[ 20%] Linking CXX static library libkeywords.a
[ 20%] Built target keywords
[ 40%] Building CXX object src/back-end/CMakeFiles/Foo.dir/main.cpp.o
/home/alfredo/repos/FlexCpp/src/back-end/main.cpp: In function ‘int main()’:
/home/alfredo/repos/FlexCpp/src/back-end/main.cpp:17:5: error: ‘yyFlexLexer’ was not declared in this scope
17 | yyFlexLexer myTest(inputStream, outputStream);
| ^~~~~~~~~~~
make[2]: *** [src/back-end/CMakeFiles/Foo.dir/build.make:80: src/back-end/CMakeFiles/Foo.dir/main.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:126: src/back-end/CMakeFiles/Foo.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
What is strange is that if I comment out the last line in main.cpp
//yyFlexLexer myTest(inputStream, outputStream);
I get the following error:
lexer.l: In member function ‘virtual int yyFlexLexer::yylex()’:
lexer.l:172:1: error: ‘yypanic’ was not declared in this scope
/usr/local/flex/include/FlexLexer.h: At global scope:
/home/alfredo/repos/FlexCpp/src/back-end/lexer.yy.cc:49:25: error: redefinition of ‘class yyFlexLexer’
49 | #define yyFlexLexer yyFlexLexer
| ^~~~~~~~~~~
/home/alfredo/repos/FlexCpp/src/back-end/lexer.yy.cc:49:25: note: previous definition of ‘class yyFlexLexer’
49 | #define yyFlexLexer yyFlexLexer
| ^~~~~~~~~~~
make[2]: *** [src/back-end/CMakeFiles/Foo.dir/build.make:94: src/back-end/CMakeFiles/Foo.dir/lexer.yy.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:126: src/back-end/CMakeFiles/Foo.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
My project directory looks like the following.
backend/
|- main.cpp (what you saw ealier)
|- CMakeLists.txt
|- keywords.hxx (contains enum tokens and struct)
|- lexer.l
|- lexer.yy.cc (this will be generated by lexer.l when we compile with c++
we set %option c++ in our lexer.l file )
Here is what my lexer.l looks like
/* we dont have an interactive session. */
%option never-interactive
/* we will handle the nodefault rule. */
%option nodefault
/* we dont want yyinput() and yyunput(). */
%option noinput nounput
/* don't include <unistd.h> we dont need it. */
%option nounistd
/*
not needed for now, in the future when users can work
with multiple files we can start using it.
*/
%option noyywrap
/*
these two options are doubled to get more detailed reports.
write performance report to stderr.
*/
%option perf-report perf-report
/* write statistics summary to stderr. */
%option verbose verbose
/* generate warning messages for mistakes. */
%option warn
/* maintain current line number in yylineno. */
%option yylineno
%option c++
/* c items that can be used in the rules section */
%{
#include <stdio.h>
#include <keywords.hxx>
int iskeyword(char * keyword);
%}
digit [0-9]
exponent [ee][-+]?{digit}+
id_start [_a-za-z]
id_after {id_start}|{digit}
/*===================rules===================*/
%%
/* a bunch of rules */
%%
/*=================user-code=================*/
#include <keywords.hxx>
/*As mentioned by Flex manual we include this if our main() is not in this file */
#include <FlexLexer.h>
int iskeyword(char * keyword){
for(int index = 0; index < NUM_KEYWORDS; index++){
if(strcmp(keywords[index].keyWord, keyword)){
return keywords[index].token;
}
}
return 0;
}
CMakeLists.cmake
cmake_minimum_required(VERSION 3.5)
project(
TestingProj
)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -isystem /usr/local/flex/include")
find_package(FLEX REQUIRED)
add_library(keywords keywords.hxx)
set_target_properties(keywords PROPERTIES LINKER_LANGUAGE CXX)
target_include_directories(keywords PUBLIC ./)
FLEX_TARGET(
gen_lexer
lexer.l
lexer.yy.cc
)
add_executable(
Foo
main.cpp
${FLEX_gen_lexer_OUTPUTS}
)
target_link_libraries(Foo PUBLIC keywords)
target_link_libraries(Foo PUBLIC ${FLEX_LIBRARIES})
If I log the following
message("FLEX_FOUND ${FLEX_FOUND}")
message("FLEX_EXECUTABLE ${FLEX_EXECUTABLE}")
message("FLEX_VERSION ${FLEX_VERSION}")
message("FLEX_LIBRARIES ${FLEX_LIBRARIES}")
message("FLEX_INCLUDE_DIRS ${FLEX_INCLUDE_DIRS}")
I get
FLEX_FOUND TRUE
FLEX_EXECUTABLE /usr/local/flex/bin/flex
FLEX_VERSION 2.6.4
FLEX_LIBRARIES /usr/local/flex/lib/libfl.so
FLEX_INCLUDE_DIRS /usr/local/flex/include
I also tired to following from the Flex Manual. Did not help.
If you want to create multiple (different) lexer classes, you use the ‘-P’ flag (or the prefix= option) to rename each yyFlexLexer to some other ‘xxFlexLexer’. You then can include <FlexLexer.h> in your other sources once per lexer class, first renaming yyFlexLexer as follows: #undef yyFlexLexer #define yyFlexLexer xxFlexLexer #include <FlexLexer.h> #undef yyFlexLexer #define yyFlexLexer zzFlexLexer #include <FlexLexer.h>
I am not sure what I am doing wrong. I have spent a lot of hours trying to get Flex and C++ to work. I was able to get Flex and plain C to work it is just c++ I am having trouble. My best guess as of right as of posting is that there is an issue with the FlexLexer.h file. Some reason other projects that ive built from source bulid a FlexLexer.h file in their /include folder. As far as im aware right now it points to the correct location of FlexLexer.h which is my case is at /usr/local/flex/include/FlexLexer.h: Or it could be something wrong with my .cmake file.
Any help will be appreciated.
I was able to solve it by first following the comment left by Tysvarev I am quoting them.
"No needs to explicitly include that file from lexer.l. But in you main.cpp you need to include FlexLexer.h, otherwise a compiler has no information about yyFlexLexer class"
Then inside a %{ ... %} block, I defined yypanic like this
void yypanic( char* errorMessage);
finally in the user code I wrote
void yypanic(char * errorMessage) {
std::cout << errorMessage << std::endl;
}