Tomcat 7 servlet needs to force a close of a long-lasting NIO connection

Question

I have a Tomcat 7 servlet accepting connections from remote clients, and keeping those connections open for hours or days if possible. So, we are using the NIO connector. The bandwidth on the physical connections may be expensive, so traffic is to be kept to an absolute minimum, therefore we have programmed the remote clients to test the connection with a very infrequent ping.

Occasionally, the servlet is told the connection is closed, but it seems that the remote clients are not told. The clients don't find out until they do a ping, at which time they can establish a new connection. We need to reduce the length of time the clients are not connected, without using more pings.

One way that works is to shut down the Tomcat server. The clients know they are disconnected immediately. Obviously we don't want to be shutting down Tomcat - my point is there must be some kind of signal making it across the normally-quiet connection.

How can I force Tomcat to send this signal? Please don't respond that I can't unless you can tell me why, with specifics, partly because I can't believe it, and mostly because I don't want to believe it. It seems strange - like not being able to hang-up the phone on someone.

Alternate question #1 - can a servlet recover a connection it has been told is closed?

Alternate question #2 - can anyone think of anything else that might help?

Answer 1

A couple of thoughts:

• I'm surprised the clients are unaware that the connection has been dropped. Are you certain that an exception is not being swallowed somewhere client side?
• With days long connections, I'd start to look at what is happening on the TCP layer. You've not mentioned any diagnostic or information...
  • Have you attempted to determine if the there is an underlying TCP/IP connection still present for these problem client/server connections?
  • How stable is the network between client and server?
  • Are their potentially misbehaving network devices, like proxies or routers involved?
• I don't really see the benefit of using the NIO connector in this context of long lived connections. Many requests per second? Yes. But just for long lived connections? No.
  • In the past I've had a bizarre issue with the NIO connector and a servlet protocol gateway (OpenAMF) that magically went away by using the default HTTP connector.
  • Test the other two connectors (HTTP, native ARP), benchmark them.
• I would also consider ditching any TCP-based protocol (like HTTP) in favor of some UDP-based approach. That way your connections would be state-less and, with a bespoke tuned payload, the packets very small.