Search code examples
c++boost-spiritboost-spirit-qi

In Boost.Spirit, why is a fusion wrapper required for a vector (wrapped in a struct), but not a variant?


I would like to understand the exact scenarios in which BOOST_FUSION_ADAPT_STRUCT is required when encapsulating structs using Boost.Spirit.

What follows are two examples. One example is a single-member struct with (only) a variant data member. This version does NOT require the BOOST_FUSION_ADAPT_STRUCT macro that wraps the struct in a Fusion container. A constructor is sufficient for Spirit to instantiate/populate the attribute based on the incoming rhs.

(Please see comments in the code for my understanding of the attribute type I think is being generated by Boost.Spirit for the rhs of the rule definitions due to the attribute collapsing rules.)

The second example is a single-memeber struct with (only) a vector data member. Even with the constructor defined to allow Spirit to populate the attribute based on the rhs, it fails to compile without BOOST_FUSION_ADAPT_STRUCT.

Why the difference? I'd like to understand why, in the first scenario, a constructor can be used to populate the attribute (the struct), whereas in the second scenario, a constructor is not sufficient and BOOST_FUSION_ADAPT_STRUCT must be used.


Examples noted above follow.

EXAMPLE 1: Variant

#include <string>
#include <vector>
#include <boost/variant.hpp>
#include <boost/spirit/include/qi.hpp>
namespace qi  = boost::spirit::qi;
typedef std::string::const_iterator It;

using intermediate = boost::variant<std::string, int>;

// Simple parser demonstrating successful build with 'works_great'
struct works_great // No need for BOOST_FUSION_ADAPT_STRUCT - whoopee!
                   // But why - even given the constructor??
{
    intermediate i;
    works_great() = default;
    works_great(intermediate i) : i{i} {}
};

// Not required for 'works_great' - constructors work just fine
//BOOST_FUSION_ADAPT_STRUCT(works_great, v)

struct parser : qi::grammar<It, works_great()>
{
    parser() : parser::base_type(works_great)
    {
        using namespace qi;
        intermediate = qi::string("test") | qi::int_;

        // rhs should have attribute of type 'variant',
        // matching the constructor
        works_great = '{' >> intermediate >> '}';
    }

  private:
    qi::rule<It, intermediate()>  intermediate;
    qi::rule<It, works_great()>   works_great;
};

int main()
{
    // The following all compiles/builds just fine
    // (I don't care about the actual runtime results).
    static const parser p;
    works_great wg;
    std::string const data {"{test}"};
    auto f(begin(data)), l(end(data));
    qi::parse(f,l,p,wg);
}

EXAMPLE 2: Vector

#include <string>
#include <vector>
#include <boost/variant.hpp>
#include <boost/spirit/include/qi.hpp>
namespace qi  = boost::spirit::qi;
typedef std::string::const_iterator It;

// We need BOOST_FUSION_ADAPT_STRUCT for this one, but not for the above.
// Constructors don't help. Only difference seems to be
// the vector (rather than variant).
struct not_so_much // not so much - unless BOOST_FUSION_ADAPT_STRUCT is used
{
    std::vector<int> s;

    // Constructors do not help here
    //not_so_much() = default;
    //not_so_much(std::vector<int> s) : s{std::move(s)} {}
};

// Required for 'not_so_much' - constructors don't work
BOOST_FUSION_ADAPT_STRUCT(not_so_much, s)

// Simple parser demonstrating successful build with 'not_so_much' -
// but only when BOOST_FUSION_ADAPT_STRUCT is used.
struct parser : qi::grammar<It, not_so_much()>
{
    parser() : parser::base_type(not_so_much)
    {
        using namespace qi;

        // Note: I know that 'eps' is required, below, to compile the 
        // single-member struct successfully

        // rhs should have attribute of type 'vector<int>',
        // matching the constructor as well...
        // but it doesn't work.
        not_so_much = eps >> (qi::int_ % "|");
    }

  private:
    qi::rule<It, not_so_much()> not_so_much;
};

int main()
{
    // The following all compiles/builds just fine
    static const parser p;
    not_so_much nm;
    std::string const data {"5|9|16"};
    auto f(begin(data)), l(end(data));
    qi::parse(f,l,p,nm);
}

Solution

  • The difference is twofold:

    • the attribute is not a container
    • the default constructor allows for implicit conversion of synthesized attribute to exposed attribute

    The latter difference, you have noticed. The first: not so much.


    The really principled answer is:

    Qi Attribute Propagation is a heuristic machine.

    Sadly, few things optimize for performance (X3 does a lot better). One of the key areas that is an exception is the incremental parsing into containers (even across multiple rules)¹.

    This makes a lot of sense (since even e.g. building strings character-by-character would be extremely slow...). But it does lead to surprises (eg. boost::spirit::qi duplicate parsing on the output, Understanding Boost.spirit's string parser)

    ¹ (actually also non-containers, but I digress. I don't think it comes into to play without semantic actions)

    Some Unnecessary Gymnastics

    You can actually change the timings at which the attribute propagations fire a bit, and do without the adaptation, though I'd advise against it: just adapting is much more consistent and self-descriptive:

    Live On Coliru

    #include <boost/spirit/include/qi.hpp>
    namespace qi = boost::spirit::qi;
    
    namespace Ast {
        using vec = std::vector<int>;
        struct not_so_much {
            vec s;
    
            not_so_much() = default;
            not_so_much(vec s) : s(std::move(s)) {}
        };
    }
    
    typedef std::string::const_iterator It;
    typedef qi::rule<It, Ast::not_so_much()> Parser;
    
    template <typename Expr> void do_test(Expr const& expression) {
        Parser const p = expression;
        Ast::not_so_much nm;
    
        std::string const data {"5|9|16"};
        It f = begin(data), l = end(data);
    
        if (qi::parse(f,l,p,nm)) {
            std::cout << "Parsed " << nm.s.size() << " elements: ";
            copy(nm.s.begin(), nm.s.end(), std::ostream_iterator<int>(std::cout, " "));
            std::cout << "\n";
        } else {
            std::cout << "Parse failed\n";
        }
    
        if (f != l)
            std::cout << "Remaining unparsed: '" << std::string(f,l) << "'\n";
    }
    
    int main() {
        using namespace qi;
        do_test(attr_cast<Ast::not_so_much, Ast::vec>(int_ % '|'));
        do_test(attr_cast<Ast::not_so_much>(int_ % '|'));
    
        do_test(as<Ast::vec>()[int_ % '|']);
    }
    

    Prints

    Parsed 3 elements: 5 9 16 
    Parsed 3 elements: 5 9 16 
    Parsed 3 elements: 5 9 16