Search code examples
javasocketstomcatsocketexceptionconnection-reset

Tomcat 7 throwing frequent java.net.SocketException: Connection reset


I am using Tomcat 7 and am getting frequent connection reset errors,

org.apache.coyote.http11.Http11Processor - Error parsing HTTP request header

java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:189)
    at java.net.SocketInputStream.read(SocketInputStream.java:121)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:516)
    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:501)
    at org.apache.coyote.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:98)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:931)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

and at times, though not frequently "Unexpected EOF read on the socket",

org.apache.coyote.http11.Http11Processor - Error parsing HTTP request header

java.io.EOFException: Unexpected EOF read on the socket
    at org.apache.coyote.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:99)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:931)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

From what i have read through, both seem to indicate that the client might have possibly abruptly closed the connection, while the server is reading it, resulting into a reset and/or EOF.

Are there any other reasons anyone can think of or have encountered. Also if both exceptions originate from the same problem, how is the frequency so varied, i get about 100 connection resets to say 5 EOF's.

Thanks


Solution

  • More investigations and Wireshark dumps reveal, other than the actual load, 2 other things, show up, one is the load balancer which does a regular heartbeat check and another is a network monitoring script that again does a frequent pings.

    So these seem to be the ones pinging and breaking the connections. So when we were monitoring a passive server which is again behind the LB and would too get pinged by the network monitoring script, but no production load. It too showed up the same exceptions that we saw on the active server with load. That confirmed the above observations. So one we can either ignore these or alter the monitoring script to wait for the responses from the server.