I'm running a high volume java web-app over a jetty 9.x server (4k-5k qps).
My servlet (MyServlet.java) is simple Java class the extends HttpServlet.
When getting the request object HttpServletRequest I invoke the getInputStream() method (as can be seen in the below stack) and then getting the below IOException.
I know that 'Broken pipe' means that the client closed the connection but when calculating the amount of time the request spent in the server I get ti 1ms and my client timeout is configured for much more.
Could this be the result of too much load? What could cause it? Is there some configuration in Jetty which I can play with that might affect this?
org.eclipse.jetty.io.EofException
at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:192)
at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:408)
at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:302)
at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:129)
at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:679)
at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:246)
at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:208)
at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:471)
at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:763)
at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:781)
at org.eclipse.jetty.server.HttpChannel.continue100(HttpChannel.java:240)
at org.eclipse.jetty.server.Request.getInputStream(Request.java:674)
at com.ta.btr.web.servlet.MyServlet.handlePostRequest(MyServlet.java:162)
at com.ta.btr.web.servlet.MyServlet.doPost(MyServlet.java:92)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:65)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:170)
... 35 more
It appears I had a problem in the load balancer which delayed the requests causing the client to close the connection (due to timeout).