Non-empty closures and the question mark: only first element is put into the AST?

Question

I'm haunted by a strange phenomenon:

Only the first in x in z: x | '<'! y? '>', where y: x (','! x)*, occurs in the resulting AST. But only if I compile the code using Antlr3 as deployed in the maven repositories. With AntlrWorks I see a correct Tree.

Is b: a a*; c: d b? d; semantically wrong? What am I doing wrong? Or is there simply an error in Antlr?

If you need a more complete example (my problem occurs in the $v2 of try_):

variables
    : annotatedVariable
        -> ^(VARIABLES annotatedVariable)
    | v='<' annotatedVariableList? '>'
        -> ^(VARIABLES[$v] annotatedVariableList?)
    ;

annotatedVariableList
    : annotatedVariable (','! annotatedVariable)*
    ;

try_
    : v='try' e1=expression 'of' v1=variables '->' e2=expression
      'catch' v2=variables '->' e3=expression
        -> ^(TRY[$v] $e1 $v1 $e2 $v2 $e3)
    ;

(Complete source and an example input file: https://gist.github.com/1004329. Only the T of in catch is present in the tree, but I see both T and R in AntlrWorks.)

Answer 1

Kay wrote:

Only the first in x in z: x | '<'! y? '>', where y: x (','! x)*, occurs in the resulting AST. But only if I compile the code using Antlr3 as deployed in the maven repositories. With AntlrWorks I see a correct Tree.

ANTLRWorks produces a parse tree, not the AST your parser rules produce (assuming you're talking about ANTLRWorks' interpreter).

A simple test shows everything works just fine (tested with ANTLR v3):

grammar T;

options {
  output=AST;
}

z : x 
  | '<'! y? '>'!
  ;

y : x (','! x)*
    ;

x : X
  ;

X : 'A'..'Z'
  ;

will parse the input:

<X,Y,Z>

into the following AST:

as you can test yourself:

import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;
import org.antlr.stringtemplate.*;

public class Main {
  public static void main(String[] args) throws Exception {
    TLexer lexer = new TLexer(new ANTLRStringStream("<X,Y,Z>"));
    TParser parser = new TParser(new CommonTokenStream(lexer));
    CommonTree tree = (CommonTree)parser.z().getTree();
    DOTTreeGenerator gen = new DOTTreeGenerator();
    StringTemplate st = gen.toDOT(tree);
    System.out.println(st);
  }
}

Kay wrote:

Is b: a a*; c: d b? d; semantically wrong?

No. But is this question related to your z: x | '<'! y? '>' rule? Sorry to say, but you're a bit vague.

Kay wrote:

What am I doing wrong?

I don't know. You'll have to post a short, self contained example here on SO, just like I posted. I'm not keen on following an external link and wading through hundreds of lines of code, sorry.