unable to understand Output of the following simple program

Question

/* A Java program to illustrate working of StringTokenizer
class:*/
import java.util.*;
public class NewClass
{
    public static void main(String args[])
    {


        System.out.println("Using Constructor 3 - ");
        StringTokenizer st3 = 
            new StringTokenizer("JAVA : Code : String", " :", true);
        while (st3.hasMoreTokens())
            System.out.println(st3.nextToken());
    }
}

Why is the output of the above program is as follows:

Using Constructor 3 - 
JAVA

:

Code

:

String

My question is why it has extra newline characters(i.e. it has blank lines after "Java", ":", "Code" etc)

Answer 1

It will become a bit clearer if we make the printing a bit more verbose:

while (st3.hasMoreTokens())
  System.out.printf("token: '%s'\n", st3.nextToken());

The output will be:

token: 'JAVA'
token: ' '
token: ':'
token: ' '
token: 'Code'
token: ' '
token: ':'
token: ' '
token: 'String'

As you see, and : are tokens. It's because every character in the specified delimiter string : are used as delimiters. And you passed true as the 3rd parameter to the constructor of StringTokenizer, and that makes it return the delimiters as tokens.

This is explained in the JavaDoc of StringTokenizer.html#StringTokenizer.