Parsing mixed values and key-value pairs with Boost.Spirit

Question

I have a simple grammar consisting of mixed variables ($(name)) and variable-value pairs ($(name:value)). I have a hand-coded recursive parser, but am interested in using it as an exercise to learn Spirit, which I'll need for more complex grammars eventually(/soon).

Anyway, the set of possible forms I'm working with (simplified from the full grammar) is:

$(variable)     // Uses simple look-up, recursion and inline replace
$(name:value)   // Inserts a new variable into the local lookup table

My current rules look something like:

typedef std::map<std::string, std::string> dictionary;

template <typename Iterator>
bool parse_vars(Iterator first, Iterator last, dictionary & vars, std::string & output)
{
    using qi::phrase_parse;
    using qi::_1;
    using ascii::char_;
    using ascii::string;
    using ascii::space;
    using phoenix::insert;

    dictionary statevars;

    typedef qi::rule<Iterator, std::string()> string_rule;
    typedef qi::rule<Iterator, std::pair<std::string, std::string>()> pair_rule;

    string_rule state = string >> ':' >> string; // Error 3
    pair_rule variable = 
    (
        char_('$') >> '(' >> 
        (
            state[insert(phoenix::ref(statevars), _1)] |
            string[output += vars[_1]] // Error 1, will eventually need to recurse
        ) >> ')'
    ); // Error 2

    bool result = phrase_parse
    (
        first, last, 
        (
            variable % ','
        ), 
        space
    );

    return r;
}

If it wasn't obvious, I have no idea how Spirit works and the docs have everything but actual explanations, so this is about an hour of throwing examples together.

The parts I particularly question are the leading char_('$') in the variable rule, but removing this causes a shift operator error (the compiler interprets '$' >> '(' as a right-shift).

When compiling, I get errors related to the state rule, particularly creating the pair, and the lookup:

1. error C2679: binary '[' : no operator found which takes a right-hand operand of type 'const boost::spirit::_1_type' (or there is no acceptable conversion)
2. error C2512: 'boost::spirit::qi::rule::rule' : no appropriate default constructor available

Changing the lookup (vars[_1]) to a simple += gives:

3. error C2665: 'boost::spirit::char_class::classify::is' : none of the 15 overloads could convert all the argument types

Error 1 seems to relate to the type (attribute?) of the _1 placeholder, but that should be a string, and is when used for printing or concatenation to the output string. 2 appears to be noise caused by 1.

Error 3, digging down the stack of template errors, seems to relate to not being able to turn the state rule into a pair, which seems odd as it almost exactly matches one of the rules from this example.

How can I modify the variable rule to properly handle both input forms?

Answer 1

A few things to note:

1. To adapt std::pair (so you can use it with maps) you should include (at least)

   #include <boost/fusion/adapted/std_pair.hpp>
   

2. It looks like you are trying to create a symbol table. You could use qi::symbols for that

3. avoid mixing output generation with parsing, it complicates matters unduly

I haven't 'fixed' all the above (due to lack of context), but I'd happy to help out with any other questions arising from those.

Here is a fixed code version staying pretty close to the OP. Edit have tested it too now, output below:

#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/fusion/adapted/std_pair.hpp>
#include <map>

namespace qi    = boost::spirit::qi;
namespace phx   = boost::phoenix;

typedef std::map<std::string, std::string> dictionary;

template <typename Iterator, typename Skipper = qi::space_type>
    struct parser : qi::grammar<Iterator, Skipper>
{
    parser(dictionary& statevars, std::string& output) : parser::base_type(start)
    {
        using namespace qi;
        using phx::insert; 

        with_initializer = +~char_(":)") >> ':' >> *~char_(")");

        simple           = +~char_(")");

        variable         = 
            "$(" >> (
                   with_initializer  [ insert(phx::ref(statevars), qi::_1) ] 
                 | simple            [ phx::ref(output) += phx::ref(statevars)[_1] ]
             ) >> ')';

        start = variable % ',';

        BOOST_SPIRIT_DEBUG_NODE(start);
        BOOST_SPIRIT_DEBUG_NODE(variable);
        BOOST_SPIRIT_DEBUG_NODE(simple);
        BOOST_SPIRIT_DEBUG_NODE(with_initializer);
    }

  private:
    qi::rule<Iterator, std::pair<std::string, std::string>(), Skipper> with_initializer;
    qi::rule<Iterator, std::string(), Skipper> simple;
    qi::rule<Iterator, Skipper> variable;
    qi::rule<Iterator, Skipper> start;
};

template <typename Iterator>
bool parse_vars(Iterator &first, Iterator last, dictionary & vars, std::string & output)
{
    parser<Iterator> p(vars, output);
    return qi::phrase_parse(first, last, p, qi::space);
}

int main()
{
    const std::string input = "$(name:default),$(var),$(name)";
    std::string::const_iterator f(input.begin());
    std::string::const_iterator l(input.end());

    std::string output;
    dictionary table;

    if (!parse_vars(f,l,table,output))
        std::cerr << "oops\n";
    if (f!=l)
        std::cerr << "Unparsed: '" << std::string(f,l) << "'\n";
    std::cout << "Output:   '" << output << "'\n";
}

Output:

Output:   'default'