How do I use the Rust parser (libsyntax) myself?

Question

I want to use the Rust parser (libsyntax) to parse a Rust file and extract information like function names out of it. I started digging in the docs and code, so my first goal is a program that prints all function names of freestanding functions in a .rs file.

The program should expand all macros before it prints the function names, so functions declared via macro aren't missed. That's why I can't write some crappy little parser by myself to do the job.

I have to admit that I'm not yet perfectly good at programming Rust, so I apologize in advance for any stupid statements in this question.

How I understood it I need to do the following steps:

1. Parse the file via the Parser struct
2. Expand macros with MacroExpander
3. ???
4. Use a Visitor to walk the AST and extract the information I need (eg. via visit_fn)

So here are my questions:

1. How do I use MacroExpander?
2. How do I walk the expanded AST with a custom visitor?

I had the idea of using a custom lint check instead of a fully fledged parser. I'm investigating this option.

If it matters, I'm using rustc 0.13.0-nightly (f168c12c5 2014-10-25 20:57:10 +0000)

Answer 1

I'm afraid I can't answer your question directly; but I can present an alternative that might help.

If all you need is the AST, you can retrieve it in JSON format using rustc -Z ast-json. Then use your favorite language (Python is great) to process the output.

You can also get pretty-printed source using rustc --pretty=(expanded|normal|typed).

For example, given this hello.rs:

fn main() {
    println!("hello world");
}

We get:

$ rustc -Z ast-json hello.rs
{"module":{"inner":null,"view_items":[{"node":{"va... (etc.)

$ rustc --pretty=normal hello.rs
#![no_std]
#[macro_use]
extern crate "std" as std;
#[prelude_import]
use std::prelude::v1::*;
fn main() { println!("hello world"); }

$ rustc --pretty=expanded hello.rs
#![no_std]
#[macro_use]
extern crate "std" as std;
#[prelude_import]
use std::prelude::v1::*;
fn main() {
    ::std::io::stdio::println_args(::std::fmt::Arguments::new({
                                                                  #[inline]
                                                                  #[allow(dead_code)]
                                                                  static __STATIC_FMTSTR:
                                                                         &'static [&'static str]
                                                                         =
                                                                      &["hello world"];
                                                                  __STATIC_FMTSTR
                                                              },
                                                              &match () {
                                                                   () => [],
                                                               }));
}

If you need more than that though, a lint plugin would be the best option. Properly handling macro expansion, config flags, the module system, and anything else that comes up is quite non-trivial. With a lint plugin, you get the type-checked AST right away without fuss. Cargo supports compiler plugins too, so your tool will fit nicely into other people's projects.