Search code examples
javaubuntufile-ioinode

java.io.FileNotFoundException and (No space left on device)

I have this Java code, which should read a 0.5M files and write after removing some useless information(I’m using Enron E-mails Dataset)

public void getInboxFiles(File directory){
    File[] usersFolders;
    File[] userFolders;
    File[] inboxFiles;
    usersFolders = directory.listFiles();
    for(File temp:usersFolders){
        userFolders = temp.listFiles();         
        for(File temp2:userFolders){                                    
            inboxFiles = temp2.listFiles();
            for(File tmp3:inboxFiles){
                if(tmp3.isDirectory())
                    continue;                       
                readNPrase(tmp3, new File("/media/ADATA SH12/datasets/parsedEnron/"+temp.getName()+tmp3.getName()+".txt"));                         
            }
        }
    }   

}

the function readNParse is:

public void readNPrase(File in,File out){
    BufferedReader br=null;
    BufferedWriter bw =null;
    try{
        br = new BufferedReader(new FileReader(in));
        bw= new BufferedWriter(new FileWriter(out));
        boolean messageContent = false;
        String line = null;
        while((line = br.readLine()) != null){              
            if(line.trim().equals(""))
                messageContent = true;
            if(messageContent && !isHeader(line) && !line.trim().equals("")){
                bw.write(line);
                bw.newLine();
            }

        }
            bw.flush();
            br.close();
            bw.close();
    } catch (IOException e) {
        e.printStackTrace();
    }finally{
        try{
            bw.close();
            br.close();
        }catch(Exception e){
            e.printStackTrace();
        }
    }
}

after running this code for 10 minuets it stops working and gave me this error:

java.lang.NullPointerException java.io.FileNotFoundException: /media/ADATA SH12/datasets/parsedEnron/causholli-m98.txt (No space left on device)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
at java.io.FileWriter.<init>(FileWriter.java:90)
at EnronMailParser.readNPrase(EnronMailParser.java:16)
at EnronMailParser.getInboxFiles(EnronMailParser.java:71)
at EnronMailParser.main(EnronMailParser.java:84)

I checked the space on the drive but there is a too much of empty space some of people said it is related to file name, and some said it is related to inode which is an os issue, I have no idea what is it and how it solve it. I am on Ubuntu 12.04.

Solution

  • As you indicated, "No space left on device" could mean that you're out of inodes on the filesystem where the new file would be stored. Unix filesystems (including Ubuntu) commonly use a data structure called an inode for each file stored in the filesystem. The number of inodes is fixed when the filesystem is created, and it limits the number of files that can be created in the filesystem. If a filesystem is out of inodes, you can't create new files even when there is free space.

    You can run df -i to see how many inodes each filesystem has and how many are free.

    Filesystems are normally created with plenty of inodes, so it's a little unusual to run out. The filesystem is likely to have a large number of very small or empty files on it. It's not uncommon to find that some program has been quietly creating empty files in some obscure directory. You should inspect the filesystem to see if there are some files you can delete. Remember that you're looking for a large number of files to delete, because you want to free a large number of inodes. It doesn't matter how big each file is.

    If the filesystem is out of inodes, there's not a lot you can do within your program to work around the issue. If your program is the one that created a bunch of empty files, then you should reconsider how your program works. Otherwise, your only choice is to create the files somewhere else where inodes aren't a problem.