Is there a Java-specific open file limit on OS X?

Question

Somehow I cannot open more than 10223 files from Java on my MacBook, while the open file limit with sysctl, launchd and ulimit is set to 65535 and I can open 62282 files from a C program.

Do you know if there is some extra limit that Java is running into, and how I can change it?

The Java code I use to test is:

import java.io.*;
import java.util.*;

class OpenFilesTest {
    private static List<InputStream> streams = new ArrayList<InputStream>();

    public static void main(String[] args) {
        for (int i = 0; true; i++) {
            FileInputStream f = null;
            try {
               f = new FileInputStream("/dev/null");
            } catch (Throwable e) {
               System.err.println(e.getMessage());
               e.printStackTrace();
               System.exit(1);
            }
            streams.add(f);
            System.out.println("We have " + (i + 1) + " InputStream's for /dev/null");
        }
    }
}

this outputs:

We have 1 InputStream's for /dev/null
We have 2 InputStream's for /dev/null
We have 3 InputStream's for /dev/null
...
We have 10221 InputStream's for /dev/null
We have 10222 InputStream's for /dev/null
We have 10223 InputStream's for /dev/null
/dev/null (Too many open files)
java.io.FileNotFoundException: /dev/null (Too many open files)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at java.io.FileInputStream.<init>(FileInputStream.java:97)
        at OpenFilesTest.main(test.java:11)

The C code I use to test:

#include <stdlib.h>
#include <stdio.h>
#include <fcntl.h>

int main(void)
{
    for (int i = 0; 1; i++)
    {
        int fd = open("/dev/null", O_RDONLY);
        if (fd == -1)
        {
            perror("Couldn't open /dev/null one more time");
            exit(EXIT_FAILURE);
        } else {
            printf("Opened /dev/null %d times\n", i + 1);
        }
    }
}

and its output:

Opened /dev/null 1 times
Opened /dev/null 2 times
Opened /dev/null 3 times
...
Opened /dev/null 62287 times
Opened /dev/null 62288 times
Opened /dev/null 62289 times
Couldn't open /dev/null one more time: Too many open files in system

Some more relevant information about the system:

$ uname -a
Darwin hostname.local 13.1.0 Darwin Kernel Version 13.1.0: Thu Jan 16 19:40:37 PST 2014; root:xnu-2422.90.20~2/RELEASE_X86_64 x86_64
$ java -version
java version "1.7.0_25"
Java(TM) SE Runtime Environment (build 1.7.0_25-b15)
Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode)

I have also tried this on Linux (Debian Wheezy 7.4) and there I can open close to 65535 filehandles also using this Java program.

The underlying use case for so many open files is a server load testing application in Scala.

Answer 1

Have a lookat the options specific for the OS X JVM. Specifically for MaxFDLimit:

-XX:- MaxFDLimit

Directs the VM to refrain from setting the file descriptor limit to the default maximum. The default behavior is to set the limit to the value specified by OPEN_MAX, which is 10240. Normally, this is the maximum number of files that a process may have open. It is possible, however, to increase this limit to a user-specified value with the sysctl utility. Under such circumstances, you may want to pass -XX:-MaxFDLimit to stop the Java VM from restricting the number of open files to 10240.