Search code examples
javaarraylistchartokenizestringtokenizer

Char count of each token in Tokenized String, Java


I'm trying to figure out if I can count the characters of each token and display that information such as:

day is tokenized and my output would be: "Day has 3 characters." and continue to do that for each token.

My last loop to print out the # of characters in each token never prints:

public static void main(String[] args) {

    Scanner sc = new Scanner(System.in);

    ArrayList<String> tokenizedInput = new ArrayList<>();
    String sentenceRetrieved;

    // getting the sentence from the user
    System.out.println("Please type a sentence containing at least 4 words, with a maximum of 8 words: ");
    sentenceRetrieved = sc.nextLine();
    StringTokenizer strTokenizer = new StringTokenizer(sentenceRetrieved);

    // checking to ensure the string has 4-8 words
    while (strTokenizer.hasMoreTokens()) {
        if (strTokenizer.countTokens() > 8) {
            System.out.println("Please re-enter a sentence with at least 4 words, and a maximum of 8");
            break;

        } else {
            while (strTokenizer.hasMoreTokens()) {
                tokenizedInput.add(strTokenizer.nextToken());
            }

            System.out.println("Thank you.");
            break;
        }
    }

    // printing out the sentence
    System.out.println("You entered: ");
    System.out.println(sentenceRetrieved);

    // print out each word given
    System.out.println("Each word in your sentence is: " + tokenizedInput);

    // count the characters in each word
    // doesn't seem to run

    int totalLength = 0;
    while (strTokenizer.hasMoreTokens()) {
        String token;
        token = sentenceRetrieved;
        token = strTokenizer.nextToken();
        totalLength += token.length();
        System.out.println("Word: " + token + " Length:" + token.length());
    }

}

}

Example of Console:

Please type a sentence containing at least 4 words, with a maximum of 8 words:

Hello there this is a test

Thank you.

You entered:

Hello there this is a test

Each word in your sentence is: [Hello, there, this, is, a, test]


Solution

  • First off, I have added the necessary imports and built a class around this main method. This should compile.

    import java.util.ArrayList;
    import java.util.Scanner;
    import java.util.StringTokenizer;
    
    public class SOQ_20200913_1
    {
    
       public static void main(String[] args) {
       
          Scanner sc = new Scanner(System.in);
       
          ArrayList<String> tokenizedInput = new ArrayList<>();
          String sentenceRetrieved;
       
        // getting the sentence from the user
          System.out.println("Please type a sentence containing at least 4 words, with a maximum of 8 words: ");
          sentenceRetrieved = sc.nextLine();
          StringTokenizer strTokenizer = new StringTokenizer(sentenceRetrieved);
       
        // checking to ensure the string has 4-8 words
          while (strTokenizer.hasMoreTokens()) {
             if (strTokenizer.countTokens() > 8) {
                System.out.println("Please re-enter a sentence with at least 4 words, and a maximum of 8");
                break;
             
             } else {
                while (strTokenizer.hasMoreTokens()) {
                   tokenizedInput.add(strTokenizer.nextToken());
                }
             
                System.out.println("Thank you.");
                break;
             }
          }
       
        // printing out the sentence
          System.out.println("You entered: ");
          System.out.println(sentenceRetrieved);
       
        // print out each word given
          System.out.println("Each word in your sentence is: " + tokenizedInput);
       
        // count the characters in each word
        // doesn't seem to run
       
          int totalLength = 0;
          while (strTokenizer.hasMoreTokens()) {
             String token;
             token = sentenceRetrieved;
             token = strTokenizer.nextToken();
             totalLength += token.length();
             System.out.println("Word: " + token + " Length:" + token.length());
          }
       
       }
    
    }
    

    Next, let's look at this working example. It seems like everything up until your final while loop (the one that counts character length) works just fine. But if you notice, the while loop before the final one will continue looping until it has no more tokens to fetch. So, after it has finished gathering all of the tokens and has no more tokens to gather, you try and create the final while loop, asking it to gather more tokens. It would not have reached the while loop until it ran out of tokens to gather!

    Finally, in order to solve this, you can simply go through the list that you added to in the second to last while loop, and simply cycle through that for your final loop!

    For example:

      int totalLength = 0;
    
      for (String each : tokenizedInput) {
    
         totalLength += each.length();
         System.out.println("Word: " + each + " Length:" + each.length());
    
      }