How do I increase the number of sockets that may be opened on ubuntu when using Mina 2.0.8? Once my server reaches around ~1700 connections, I get this in my logs:
2013-01-24 04:21:49,465 [NioSocketAcceptor-1] WARN o.a.m.util.DefaultExceptionMonitor - Unexpected exception.
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) ~[na:1.6.0_20]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163) ~[na:1.6.0_20]
at org.apache.mina.transport.socket.nio.NioSocketAcceptor.accept(NioSocketAcceptor.java:159) ~[mina-core-2.0.8-SNAPSHOT.jar:na]
at org.apache.mina.transport.socket.nio.NioSocketAcceptor.accept(NioSocketAcceptor.java:50) ~[mina-core-2.0.8-SNAPSHOT.jar:na]
at org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.processHandles(AbstractPollingIoAcceptor.java:506) ~[mina-core-2.0.8-SNAPSHOT.jar:na]
at org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:447) ~[mina-core-2.0.8-SNAPSHOT.jar:na]
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) [mina-core-2.0.8-SNAPSHOT.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [na:1.6.0_20]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [na:1.6.0_20]
at java.lang.Thread.run(Thread.java:636) [na:1.6.0_20]
How is this possible when my file handles are set to a max of 331278 for hard and soft? Running "ulimit -n" reports
331287
and running "sysctl fs.file-nr" reports
fs.file-nr = 1728 0 331287
Contents of /etc/security/limits.conf file
* hard nofile 331287 * soft nofile 331287
Two things seem to have fixed the issue I was having above:
Increasing the file handles for the "root" user specifically since its not included when using "*" in limits.conf.
Editing my init.d script to include force setting the ulimit for the process being created. There is apparently a known issue in Ubuntu dealing with process daemon file handles. This link has the details: http://www.jayway.com/2012/02/11/how-to-really-fix-the-too-many-open-files-problem-for-tomcat-in-ubuntu/