Search code examples
clanglibtooling

How VisitNamedDecl and VisitCXXRecordDecl are called by RecursiveASTVisitor?


In RecursiveASTVisitors doc, there is are no virtual methods of name VisitNamedDecl, VisitCXXRecordDecl ..etc. So how they are called automatically called ?


Solution

  • There are two things that make the design of RecursiveASTVisitor difficult to understand:

    1. It uses the curiously recurring template pattern (CRTP) to dispatch to the user-provided methods, rather than the more conventional (but possibly slower) technique of using C++ virtual methods.

    2. The vast majority of the overrideable methods are declared using a somewhat complex system of macros and multiply-included header files, the latter being generated by a special tool during the build. Among the consequences is that doxygen, which generates the published documentation you (and I) linked to, cannot see them.

    The dispatch design is explained somewhat in the Detailed Description section of the documentation, but I'll provide my own explanation to fill in some missing details.

    Due to the second issue, I will be using fragments of preprocessor output. For concreteness, I am preprocessing deleted-ctor.cc, the code I recently posted to this answer, but the any preprocessesed form of RecursiveASTVisitor.h should match the parts I am showing.

    The visitor dispatch mechanism

    Let's focus on VisitNamedDecl since it was mentioned in the question. The template class definition, after preprocessing (and some indentation using the indent tool and other minor edits for readability), looks like:

    template <typename Derived>
    class RecursiveASTVisitor {
    public:
      ...
      Derived &getDerived() { return *static_cast<Derived *>(this); }
      ...
      bool TraverseDecl(Decl *D);
      bool TraverseLabelDecl(LabelDecl *D);
      ...
      bool WalkUpFromDecl(Decl *D)
      {
        return getDerived().VisitDecl(D);
      }
    
      bool WalkUpFromNamedDecl(NamedDecl *D)
      {
        if (!getDerived().WalkUpFromDecl(D))
          return false;
        if (!getDerived().VisitNamedDecl(D))
          return false;
        return true;
      }
    
      bool VisitNamedDecl(NamedDecl *D)
      {
        return true;
      }
    
      bool WalkUpFromLabelDecl(LabelDecl *D)
      {
        if (!getDerived().WalkUpFromNamedDecl(D))
          return false;
        if (!getDerived().VisitLabelDecl(D))
          return false;
        return true;
      }
      ...
    };
    

    The Traverse methods are the entry points. They call the WalkUpFrom methods, which "walk up" the C++ class hierarchy and invoke the Visit methods for classes of which the AST node in question is an instance.

    The getDerived method is a key element of the CRTP mechanism. By invoking methods on it, instead of this directly, any method defined in Derived will (statically) override the corresponding base class method.

    The superclass TraverseDecl method does a dynamic dispatch based on the type of its argument:

    template <typename Derived>
    bool RecursiveASTVisitor<Derived>::TraverseDecl(Decl *D)
    {
      ...
      switch (D->getKind()) {
        ...
        case Decl::Label:
          if (!getDerived().TraverseLabelDecl(static_cast<LabelDecl *>(D)))
            return false;
          break;
        ...
      }
      return true;
    }
    

    I've chosen to focus on LabelDecl because it is a concrete class derived from the abstract NamedDecl. Its Traverse method looks like this:

    template <typename Derived>
    bool RecursiveASTVisitor<Derived>::TraverseLabelDecl(LabelDecl *D)
    {
      bool ShouldVisitChildren = true;
      bool ReturnValue = true;
    
      if (!getDerived().shouldTraversePostOrder())
        if (!getDerived().WalkUpFromLabelDecl(D))
          return false;
    
      if (ReturnValue && ShouldVisitChildren)
        if (!getDerived().TraverseDeclContextHelper(dyn_cast<DeclContext>(D)))
          return false;
    
      if (ReturnValue) {
        for (auto *I : D->attrs())
          if (!getDerived().getDerived().TraverseAttr(I))
            return false;
      }
    
      if (ReturnValue && getDerived().shouldTraversePostOrder())
        if (!getDerived().WalkUpFromLabelDecl(D))
          return false;
    
      return ReturnValue;
    }
    

    The idea is it calls WalkUpFrom on the current node, and Traverse on the child nodes. There are two calls to WalkUpFrom because the visitor can operate in either pre-order or post-order mode. Again, all calls go through getDerived, so can be overridden.

    The macro metaprogramming system

    Going back to the definition of RecursiveASTVisitor, the first part of the class is defined directly in RecursiveASTVisitor.h.

    But then we get to this part of the header file, which declares the Traverse methods for the Decl hierarchy:

    // ---- Methods on Decls ----
    
    // Declare Traverse*() for all concrete Decl classes.
    #define ABSTRACT_DECL(DECL)
    #define DECL(CLASS, BASE) bool Traverse##CLASS##Decl(CLASS##Decl *D);
    #include "clang/AST/DeclNodes.inc"
      // The above header #undefs ABSTRACT_DECL and DECL upon exit.
    

    DeclNodes.inc is a file generated during the build from DeclNodes.td using a tool called TableGen. In DeclNodes.inc, we find (among many other things):

    #ifndef NAMED
    #  define NAMED(Type, Base) DECL(Type, Base)
    #endif
    ABSTRACT_DECL(NAMED(Named, Decl))
    ...
    #ifndef LABEL
    #  define LABEL(Type, Base) NAMED(Type, Base)
    #endif
    LABEL(Label, NamedDecl)
    

    Given the definitions of ABSTRACT_DECL and DECL that appear just before the #include, we see that ABSTRACT_DECL(NAMED(Named, Decl)) expands to nothing, and:

    LABEL(Label, NamedDecl)         expands to
    NAMED(Label, NamedDecl)         which expands to
    DECL(Label, NamedDecl)          which expands to
    bool TraverseLabelDecl(LabelDecl *D);
    

    Next in RecursiveASTVisitor.h we have:

      // Define WalkUpFrom*() and empty Visit*() for all Decl classes.
      bool WalkUpFromDecl(Decl *D) { return getDerived().VisitDecl(D); }
      bool VisitDecl(Decl *D) { return true; }
    #define DECL(CLASS, BASE)                                                      \
      bool WalkUpFrom##CLASS##Decl(CLASS##Decl *D) {                               \
        TRY_TO(WalkUpFrom##BASE(D));                                               \
        TRY_TO(Visit##CLASS##Decl(D));                                             \
        return true;                                                               \
      }                                                                            \
      bool Visit##CLASS##Decl(CLASS##Decl *D) { return true; }
    #include "clang/AST/DeclNodes.inc"
    

    After concretely defining WalkUpFromDecl and VisitDecl, this defines WalkUpFrom and Visit for the subclasses. I won't go through all the macro expansion details for these.

    Finally, TraverseLabelDecl comes from this macro:

    // This macro makes available a variable D, the passed-in decl.
    #define DEF_TRAVERSE_DECL(DECL, CODE)                                          \
      template <typename Derived>                                                  \
      bool RecursiveASTVisitor<Derived>::Traverse##DECL(DECL *D) {                 \
        bool ShouldVisitChildren = true;                                           \
        bool ReturnValue = true;                                                   \
        if (!getDerived().shouldTraversePostOrder())                               \
          TRY_TO(WalkUpFrom##DECL(D));                                             \
        { CODE; }                                                                  \
        if (ReturnValue && ShouldVisitChildren)                                    \
          TRY_TO(TraverseDeclContextHelper(dyn_cast<DeclContext>(D)));             \
        if (ReturnValue) {                                                         \
          /* Visit any attributes attached to this declaration. */                 \
          for (auto *I : D->attrs())                                               \
            TRY_TO(getDerived().TraverseAttr(I));                                  \
        }                                                                          \
        if (ReturnValue && getDerived().shouldTraversePostOrder())                 \
          TRY_TO(WalkUpFrom##DECL(D));                                             \
        return ReturnValue;                                                        \
      }
    

    and this use of it:

    DEF_TRAVERSE_DECL(LabelDecl, {// There is no code in a LabelDecl.
                                 })
    

    Other Traverse methods have (hand-written) code inside the braces that calls Traverse on child nodes.

    The result of all this mechanism is the compiler sees Traverse, WalkUpFrom, and Visit methods for all AST classes, much of it generated automatically, but doxygen can't see most of it.