Forbid token to parse extra whitespace

Question

Now idea is like this: in Java for octalIntegerLiteral I have a rule

octalNumeral, (integerTypeSuffix optional)

But I want to get a numbers as token, so I used:

octalNumeral javaToken, (integerTypeSuffix optional)

The problem is that it starts to parse strings like: 0777 L. Can this be easily solved, or I should just deal with it is subclass?

Answer 1

Seems that #javaToken does two things at once: trimming whitespace and creating tokens. Split this behavior into two separate parsers that you can use individually. To keep old code working you can then redefine #javaToken from the new basic parsers as:

PPParser>>#javaToken
    ^ self javaPlainToken javaWhitespace

Then your number parser would look like this:

(octalNumber javaPlainToken , integerTypeSuffix optional) javaWhitespace