Search code examples
c++bisonbisonc++

Bison C++ get name of token - yytname_ is private


I'm attempting to get the name of a token in C++ Bison: E.g. %token <int> TPLUS "+" TMINUS "-" TMUL "*" TDIV "/"

However in the C++ variant of Bison, %token-table does not do anything.

I have noticed that there is a token mapping in the generated bison code: const char* const parser::yytname_[]; however it is private.

Does anyone have any idea of how to get the names of these tokens in C++ Bison? Is this even possible using the C++ variant?

I know this question was asked here, but there is no accepted answer.

Using version 3.4.3


Solution

  • First, ensure that you have a recent version of Bison (I believe the minimum is v3.6, but the v3.7 versions have several useful bug fixes).

    That will generate a static member function named symbol_name with one of the following prototypes. Note that `token_symbol_kind is an internal token number, not the value produced by the scanner. (See below)

    const char* yy::parser::symbol_name(token_symbol_kind)  (1)
    std::string yy::parser::symbol_name(token_symbol_kind)  (2)
    
    (1) if %define parse.error custom|detailed
    (2) if %define parse.error verbose
    

    If the option parse.error is not defined, then you can still get the symbol_name definition by using the deprecated %token-table directive, or if you arrange for #define YYDEBUG to be inserted in the generated code (see the -d flag, for example). I have no idea why the return type differs depending on the definition of parse.error.

    As noted, the argument to symbol_name is a bison internal token number, rather than the token type returned by the scanner. You can get the internal token number for a token type using the parser member class by_kind, with an expression like yy::parser::by_kind(yy::parser::token::«token-type-name»).type_get().