context in visitor method returns null for sub expressions

Question

so: (1) I'm just a beginner at this whole creating a compiler. (2) I will try hard and put in relevant bits about this, putting in all details would make this question just too long. I will monitor this question constantly for any details you guys need.

I'm trying to write a compiler for a simple expression/SQL like language, it should look/work like this:

Step 1: [Account Number] Greater Than [1];
Step Two: [Gender] Equals [M];
Step Last: Step 1 And Step Two;

So it's something like : Step<Name>: [Field Name] Operator [Constant]; The constant can be numbers, if they aren't they'll be handled as strings. (So you could do [This is a Constant Value])

And then the last step has to be something that combines a bunch of other steps into a tree. Execution begins from the last step and all other steps are ignored.

---

BUT, The problem is .... I'm just trying to get one simple hard coded expression to work and I keep getting a null for the Field Name inside one of the visit methods.

All I'm trying to is make this ONE STEP program work:

Step 1: [Account Number] Equals [1];

This is the grammar file:

Population.g4

grammar Population;

@parser::members
{
    protected const int EOF = Eof;
}
@lexer::members
{
    protected const int EOF = Eof;
    protected const int HIDDEN = Hidden;
} 

/* Parser Rules */

prog: step;

step           : 'Step' .+ ':' field op=('Equals'|'Greater Than') numeric_const ';';

field          : FLD_REF;

numeric_const : 
                '[' INT ']' # NumericConst
                ;

/* Lexer Rules */

INT         : [0-9]+;
FLD_REF     : '[' .+ ']';
WS
    :   (' ' | '\r' | '\n') -> channel(HIDDEN)
    ;

PopulationVisitor.cs

public class PopulationVisitor : PopulationBaseVisitor<ICompilableExpression>
{
    public override ICompilableExpression VisitStep(PopulationParser.StepContext sCTX)
    {
        //**THIS IS THE PROBLEM**** ctx is always null
        PopulationParser.FieldContext ctx = sCTX.field();
        return base.VisitStep(sCTX);
    }
    public override ICompilableExpression 
                  VisitNumericConst(PopulationParser.NumericConstContext context)
    {
        int val = int.Parse(context.GetText();
        //I then use val to build an ICompilableExpression
    }
    public override ICompilableExpression 
                  VisitField(PopulationParser.FieldContext context)
    {
       //this ends up never being called.
    }
}

Why is the ctx null? But somehow if I change the grammar to this:

step : 'Step' .+ ':' FLD_REF op=('Equals'|'Greater Than') numeric_const ';';

NOW:

public override ICompilableExpression VisitStep(PopulationParser.StepContext context)
{
    var ctx = context.FLD_REF();
    //NOW, this returns an instance of ITerminalNode
    //and ctx is not null
    return base.VisitStep(context);
}

Why does the thing work if I use a lexer token but not a grammar rule that uses the SAME lex token!?

I bet there is something just basic in my understanding of all of this that is flawed. I would sincerely appreciate clues and/or help.

Answer 1

The .+ is greedy, i.e. '[' .+ ']' grabs the first '[' and the last ']' (and everything in between, of course).

Try either '[' .+? ']' or '[' ~[\]]+ ']'