StringTokenizer vs split() in a program

Question

I have to capitalize the first character of every word in a sentence. First character of the first word is in uppercase, and the sentence is to be terminated by a full stop or a question mark only. Then I have to count all the vowels and the consonants in every word of the sentence.

Here's my code:

import java.io.*;
import java.util.*;
class ISC_Prac_2015
{
String s;
ISC_Prac_2015()
{
    s="";
}

void input()throws IOException
{
    System.out.println("Enter a sentence :");
    BufferedReader buf=new BufferedReader(new InputStreamReader(System.in));
    s=buf.readLine();
    if(!((s.charAt(s.length()-1)=='.'||s.charAt(s.length()-1)=='?')))
    {
        System.out.println("Invalid input.");
        System.exit(0);
    }
}

char changeToUpper(char c)
{
    return Character.toUpperCase(c);
}

boolean isVowel(char c)
{
    if(c=='a'||c=='e'||c=='i'||c=='o'||c=='u'||c=='A'||c=='E'||c=='I'||c=='O'||c=='U')
        return true;
    else
        return false;
}

void main()throws IOException
{
    input();
    String s2=s.substring(0,s.length()-1);
    StringBuffer s3=new StringBuffer(s2);
    for(int x=0;x<s3.length()-1;x++)
        if(s3.charAt(x)==' ')
            s3.setCharAt(x+1,changeToUpper(s3.charAt(x+1)));
    String s4=s3.toString();
    System.out.println("\n"+s4);
    StringTokenizer st=new StringTokenizer(s4);
    String a[]=new String[st.countTokens()];
    for(int x=0;x<st.countTokens();x++)
    a[x]=st.nextToken();
    /*replace the 4 lines above with String a[]=s4.split(" ");
      and the program works, but why?*/
    System.out.println("Word\t\t"+"Vowels\t\t"+"Consonants");
    int vowel=0,consonant=0;
    for(int x=0;x<a.length;x++)
    {
        for(int y=0;y<a[x].length();y++)
            if(isVowel(a[x].charAt(y)))
                vowel++;
            else
                consonant++;
        if(a[x].length()<8)
            System.out.println(a[x]+"\t\t"+vowel+"\t\t"+consonant);
        else if(a[x].length()>=8&&a[x].length()<16)
            System.out.println(a[x]+"\t"+vowel+"\t\t"+consonant);
        else
            System.out.println(a[x]+vowel+"\t\t"+consonant);
    }
}
}

The class name is peculiar because it was a question asked in my computer exam.

The program is somewhat working but I'm getting a NullPointerException for some reason on the inner for loop in main(). But when, as I have commented in the program, if I use split(" ") function, the program works correctly.

But why? Isn't StringTokenizer doing the same thing as split(" ")?

Answer 1

The API documentation for StringTokenizer's countTokens method says:

Calculates the number of times that this tokenizer's nextToken method can be called before it generates an exception. The current position is not advanced.

You're calling it within the for loop as if you expect it's going to stay constant, but it's reduced by 1 each time you advance by another token. So if your string has 4 tokens you only get the first two. You're left with nulls in your string array.

You can check this kind of thing out by using a debugger or adding printlns to your code.

There are several solutions. You can fix the code to store the token count upfront rather than keep calling the method. Better, the hasMoreTokens method is there for exactly this situation, use it. Or you could follow the advice in the documentation:

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.

Check out the documentation. When something doesn't behave like you assume it should, scrutinize the documentation to see if your assumptions are really valid.