Search code examples
compiler-errorsg++bisonminizinc

Compilation by g++ of parser code created by bison fails because `yytokentype` enum values are not visible


I'm trying to compile chuffed on Linux with

  • gcc-c++-10.3.1-1.fc33.x86_64
  • flex-2.6.4-5.fc33.x86_64
  • bison-3.6.4-3.fc33.x86_64

This is a CMAKE managed installation.

As desribed in the README, starting in the toplevel directory of the distribution, I run:

mkdir build
cd build
cmake ..
cmake --build . --verbose

At this point, errors appear. The creation of the parser code seems to work. Then /usr/bin/c++ is invoked with the option -std=gnu++11. It terminates with:

parser.tab.cpp: In function ‘int yyparse(void*)’:
parser.tab.cpp:1963:12: error: ‘YYEMPTY’ was not declared in this scope
 1963 |   yychar = YYEMPTY; /* Cause a token to be read.  */
      |            ^~~~~~~
parser.tab.cpp:2077:17: error: ‘YYEOF’ was not declared in this scope
 2077 |   if (yychar <= YYEOF)
      |                 ^~~~~
parser.tab.cpp:2083:22: error: ‘YYerror’ was not declared in this scope; did you mean ‘yyerror’?
 2083 |   else if (yychar == YYerror)
      |                      ^~~~~~~
      |                      yyerror
parser.tab.cpp:2089:16: error: ‘YYUNDEF’ was not declared in this scope; did you mean ‘YYUSE’?
 2089 |       yychar = YYUNDEF;
      |                ^~~~~~~
      |                YYUSE
parser.tab.cpp:3650:21: error: ‘YYEOF’ was not declared in this scope
 3650 |       if (yychar <= YYEOF)
      |                     ^~~~~

In the source, the constants which the compiler is looking for are defined in an enum:

/* Token kinds.  */
#ifndef YYTOKENTYPE
# define YYTOKENTYPE
  enum yytokentype
  {
    YYEMPTY = -2,
    YYEOF = 0,                     /* "end of file"  */
    YYerror = 256,                 /* error  */
    YYUNDEF = 257,                 /* "invalid token"  */
    INT_LIT = 258,                 /* INT_LIT  */
    BOOL_LIT = 259,                /* BOOL_LIT  */
    FLOAT_LIT = 260,               /* FLOAT_LIT  */
    ID = 261,                      /* ID  */
    STRING_LIT = 262,              /* STRING_LIT  */
...
    WHERE = 301                    /* WHERE  */
  };
  typedef enum yytokentype yytoken_kind_t;
#endif

and used for example like this in yyparse(void *parm):

  YYDPRINTF ((stderr, "Starting parse\n"));

  yychar = YYEMPTY; /* Cause a token to be read.  */
  goto yysetstate;

At first sight, there seems to be no cause for the compiler to complain. However, there might be something specific about the C++ version or the syntax used here. My C++ is very rusty, and this might be something straightforward.

I tried to change the line to

  yychar = yytokentype::YYEMPTY; /* Cause a token to be read.  */

but then I get:

parser.tab.cpp:1963:25: error: ‘YYEMPTY’ is not a member of ‘yytokentype’
 1963 |   yychar = yytokentype::YYEMPTY; /* Cause a token to be read.  */

That's even more perplexing.


Solution

  • At the suggestion of n-1-8e9-wheres-my-share-m,

    assuming

    CHUFFED_TOPDIR=... # toplevel of the distribution
    

    we find that there is an existing parser.tab.h created with old-ish GNU Bison 3.0.4:

    "$CHUFFED_TOPDIR"/chuffed/flatzinc/parser.tab.h
    

    This file is indeed included during compilation, as can be ascertained when compiling like so, which causes the inclusion tree to be printed:

    cd "$CHUFFED_TOPDIR"
    rm -rf build/
    mkdir build
    cd build/
    cmake -E env CXXFLAGS="-H" cmake .. # using cmake to run cmake
    cmake --build . --verbose
    

    Note

    The usual command line for running cmake would be:

    cmake -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR ..
    

    Definitely consider switching on debug mode for a first try. This is important for running the examples, which do not check their commandline arguments and segfault if these are missing 😨. With debugging on, you at least get assertion error messages:

    cmake -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR \
          -DCMAKE_BUILD_TYPE=Debug ..
    

    So, altogether

    cmake -E env CXXFLAGS="-H" \
       cmake -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR
             -DCMAKE_BUILD_TYPE=Debug ..
    

    Thus, try removing the parser.tab.h and, to make sure, the accompanying parser.tab.cpp:

    mv "$CHUFFED_TOPDIR"/chuffed/flatzinc/parser.tab.h \
       "$CHUFFED_TOPDIR"/chuffed/flatzinc/parser.tab.h.bak
    
    mv "$CHUFFED_TOPDIR"/chuffed/flatzinc/parser.tab.cpp \
       "$CHUFFED_TOPDIR"/chuffed/flatzinc/parser.tab.cpp.bak
    

    A similar problem may exist for lexer.yy.cpp, and thus:

    mv "$CHUFFED_TOPDIR"/chuffed/flatzinc/lexer.yy.cpp \
       "$CHUFFED_TOPDIR"/chuffed/flatzinc/lexer.yy.cpp.bak
    

    Note It is always good to take a look at

    "$CHUFFED_TOPDIR"/build/CMakeFiles/CMakeError.log
    

    but in my case it just says that looking for the pthread library failed, at least initially.

    Compilation succeeds now. Bison output can be found as:

    "$CHUFFED_TOPDIR"/build/parser.tab.cpp
    

    as well as:

    "$CHUFFED_TOPDIR"/build/chuffed/flatzinc/parser.tab.h
    

    Note that in the CMakeLists.txt file, we find this code:

    find_package(BISON)
    if(BISON_FOUND)
      file(MAKE_DIRECTORY ${PROJECT_BINARY_DIR}/chuffed/flatzinc)
      bison_target(FZNParser
        ${PROJECT_SOURCE_DIR}/chuffed/flatzinc/parser.yxx
        ${PROJECT_BINARY_DIR}/parser.tab.cpp
        DEFINES_FILE ${PROJECT_BINARY_DIR}/chuffed/flatzinc/parser.tab.h
        COMPILE_FLAGS "-l"
      )
    else()
      message(WARNING "Bison cannot be run. Using cached file, which may be out of date.")
      set(BISON_FZNParser_OUTPUTS
        ${PROJECT_SOURCE_DIR}/chuffed/flatzinc/parser.tab.cpp
        ${PROJECT_SOURCE_DIR}/chuffed/flatzinc/parser.tab.h
      )
    endif()
    
    find_package(FLEX)
    if(FLEX_FOUND)
      flex_target(FZNLexer
        ${PROJECT_SOURCE_DIR}/chuffed/flatzinc/lexer.lxx
        ${PROJECT_BINARY_DIR}/lexer.yy.cpp
        COMPILE_FLAGS "-L"
      )
      add_flex_bison_dependency(FZNLexer FZNParser)
    else()
      message(WARNING "Flex cannot be run. Using cached file, which may be out of date.")
      set(FLEX_FZNLexer_OUTPUTS ${PROJECT_SOURCE_DIR}/chuffed/flatzinc/lexer.yy.cpp)
    endif()
    

    I suppose the provided parser.tab.h should be automatically skipped if bison can be found on the compiling system. I'm not sure what's going wrong here.