Failed to add to tree map that is inside another map (to create an inverted index)

Question

I am working in creating inverted index for list of words in java. Basically it creates a list for each word contains the document index that word appear on associated with frequency of word in that document, the desired output should be like this:

[word1:[FileNo:frequency],[FileNo:frequency],[FileNo:frequency],word2:[FileNo:frequency],[FileNo:frequency]...etc] 

Here is the code:

package assigenment2;
import java.io.*;

import java.util.*;

public class invertedIndex {
    public static Map<String, Map<Integer,Integer>> wordTodocumentMap;
    public static BufferedReader buffer;
    public static BufferedReader br;
    public static BufferedReader reader;
    public static List<String> files = new ArrayList<String>();
    public static List<String>[] tokens; 

public static void main(String[] args) throws IOException {
    //read the token file and store the token in list
    String tokensPath="/Users/Manal/Documents/workspace/Information Retrieval/tokens.txt";
    int k=0;
    String[] tokens = new String[8500];
    String sCurrentLine;

    try
    {
        FileReader fr=new FileReader(tokensPath);
        BufferedReader br= new BufferedReader(fr);

        while ((sCurrentLine = br.readLine()) != null)
        {
            tokens[k]=sCurrentLine;
            k++;
        }

        System.out.println("the number of token are:"+k+" words");
        br.close();

    }
    catch(Exception ex)
    {System.out.println(ex);}

Until there it works correctly, I believe that the problem is in the manipulating the nested map in the following part:

    TreeMap<Integer,Integer> documentToCount = new TreeMap<Integer,Integer>();

    //read files    
    System.out.print("Enter the path of files you want to process:\n");
    Scanner InputPath = new Scanner(System.in);
    String cranfield = InputPath.nextLine();
    File cranfieldFiles = new File(cranfield);  

        for (File file: cranfieldFiles.listFiles())
        {
            int fileno = files.indexOf(file.getPath());

            if (fileno == -1) //the current file isn't in the files list \
                {
                files.add(file.getPath());// add file to the files list
                fileno = files.size() - 1;//the index of file will start from 0 to size-1
                 }
             int frequency = 0;
             BufferedReader reader = new BufferedReader(new FileReader(file));
            for (String line = reader.readLine(); line != null; line = reader.readLine()) 
            {
                for (String _word : line.split(" ")) 
                {
                    String word = _word.toLowerCase();
                    if (Arrays.asList(tokens).contains(word))
                            if (wordTodocumentMap.get(word) == null)//check whether word is new word
                                    {
                                    documentToCount = new TreeMap<Integer,Integer>();
                                    wordTodocumentMap.put(word, documentToCount);
                                    }
                                documentToCount.put(fileno, frequency+1);//add the location and frequency
                                }   
                }
        }
        reader.close();
    }
}

The error I get is:

Exception in thread "main" java.lang.NullPointerException
at assigenment2.invertedIndex.main(invertedIndex.java:65)

Answer 1

You’re never instantiating wordTodocumentMap, so it remains null throughout. Therefore the line if (wordTodocumentMap.get(word) == null)//check whether word is new word throws a NullPointerException when you do .get(), that is, before you have anything to compare to null. One possible solution is to instantiate the map in the declaration:

public static Map<String, Map<Integer,Integer>> wordTodocumentMap = new HashMap<>();

There may be other problems in your code, but this should get you a step further.