lexical analysis gives only one output?

Question

I tested this example(lexe.java), but it gave me only one output.

I gave this text as a reader:

public class LexeTest{
 private int a = 14;
}

And the nextToken() function is :

 public Category nextToken () {
        if (inp.findWithinHorizon (tokenPat, 0) == null)
            return Category.EOF;
        else {
            lastLexeme = inp.match ().group (0);
            if (inp.match ().start (1) != -1)
                return nextToken ();
            else if (inp.match ().start (2) != -1) 
                return Category.IDENT;
            else if (inp.match ().start (3) != -1)
                return Category.NUMERAL;
            Category result = tokenMap.get (lastLexeme);
            if (result == null)
                return Category.ERROR;
            else
                return result;
        }
    }

Isdie the main method:

System.out.println(lexeObject.nextToken());

output is :

IDENT

Why? but the textfile contains multiple keywords? Anyone know what's the problem?

Answer 1

Sorry to say that, but the problem seems to be your reading ability. Here is a part of the very first comment of the example you linked to (emphasis added by me):

A Lexer produces a stream of tokens taken from an arbitrary Reader supplied to the constructor, in response to repeated calls to the Lexer's nextToken routine.