Lexical Analysis for import and method calls in java

Question

How are import statements and method calls tokenized in java's lexical analysis. For example:

import java.util.Scanner

is this seen as import, java, util, Scanner(4 tokens) or import, java.util.Scanner(2 tokens)

In the same line of thought, in:

Scanner input = new Scanner(System.in);
int x = input.nextInt(); 

is input.nextInt() seen as input, nextInt() (2 tokens) or input.nextInt() (1 token)

Answer 1

Lexical analysis is described in Chapter 3 of the JLS.

This means in your first example, it would be tokenized as

keyword: import 
whitespace 
identifier: java 
seperator: . 
identifier: util 
seperator: .
identifier: Scanner
seperator: ;

So neither 2 nor 4, but 8 tokens (since whitespace and seperators are tokens according to the JLS).

Similary, input.nextInt(); is 6 tokens, since both ( and ) are one token each (See JLS § 3.11).