How to get (some) filenames within a directory efficiently

Question

I have a directory with 20M+ files in it. If I try

folder.list();

It'll take me about 5~10min to get the file (sometimes even more). I don't need all file names, just a handful everytime.

In Linux if I try:

• ls -l | head -n 100: it'll take forever
• ls -f | head -n 100: response is obtained in a few seconds.

So I can list files quickly if I use ProcessBuilder and run something like ls -f | head -n 100

Is there a native way for listing a fixed number of files within a directory without requiring to using ProcessBuilder?

Answer 1

Yes, there is way using Java NIO.2 and the class DirectoryStream. This class implements a lazy iterable over the entries of a directory. You can obtain an instance of DirectoryStream<Path> using Files.newDirectoryStream(path) (and you can obtain an instance of Path with the static factories Paths.get).

If you are using Java 8, an even simpler solution would be to use Files.list():

Return a lazily populated Stream, the elements of which are the entries in the directory. The listing is not recursive.

You could then have

List<String> fileNames = Files.list(path)
                              .map(Path::getFileName)
                              .map(Path::toString)
                              .limit(100)
                              .collect(Collectors.toList());

to retrieve the 100 file names of the given path.