Search code examples
rustrust-macros

Decl macro not matching hardcoded value once token is consumed as literal?


I'm writing a TT muncher decl macro for parsing regular expressions.

An example of the syntax I'm using is as follows:

regex!(a b c ( a b c )) // equivalent to r"abc(abc)"
regex!('n' s o m e t h i n g) // equivalent to r"\nsomething"

For this I need to consume either $char:ident or $char:literal to match a character or escaped character respectively.

I've encountered an issue, however. It seems that once a token is matched as a literal, it can no longer be matched against a hardcoded value.

Here is a reproducable example on rustc 1.69.0:

macro_rules! is_a {
    ('a') => {"Found literal a"};
    ($any:literal) => {"Not literal a?"};
    (a) => {"Found ident a"};
    ($any:ident) => {"Not ident a?"};
}

macro_rules! to_lit {
    ($any:literal) => {is_a!($any)};
}

macro_rules! to_ident {
    ($any:ident) => {is_a!($any)};
}

fn test() {

    println!("{}", is_a!('a')); // Found literal a
    println!("{}", is_a!(a)); // Found ident a
    println!("{}", to_lit!('a')); // Not literal a?
    println!("{}", to_ident!(a)); // Found ident a
}

I can get around this by using a different syntax, but I'm very confused why it's happening in the first place. Can anyone help me understand?


Solution

  • From the reference:

    Forwarding a matched fragment

    When forwarding a matched fragment to another macro-by-example, matchers in the second macro will see an opaque AST of the fragment type. The second macro can't use literal tokens to match the fragments in the matcher, only a fragment specifier of the same type. The ident, lifetime, and tt fragment types are an exception, and can be matched by literal tokens. The following illustrates this restriction:

    macro_rules! foo {
        ($l:expr) => { bar!($l); }
    // ERROR:               ^^ no rules expected this token in macro call
    }
    
    macro_rules! bar {
        (3) => {}
    }
    
    foo!(3);
    

    The following illustrates how tokens can be directly matched after matching a tt fragment:

    // compiles OK
    macro_rules! foo {
        ($l:tt) => { bar!($l); }
    }
    
    macro_rules! bar {
       (3) => {}
    }
    
    foo!(3);
    

    There is no workaround sadly; the only solution is to use proc-macros.