When the backend tomcat server receives the request, and if no worker threads are available, the request is queued. I have a Frontend that waits for (say) 30s before it displays timeout and terminates the connection. If the frontend times-out when the request is in the queue, even if the connection is terminated, the request is still processed by the server when the request reaches the top of the queue and a worker thread becomes available. But this processing is redundant as the frontend has already timed-out.
Furthermore, the request stays in the queue till it gets a chance to be processed, essentially blocking the queue for newer requests for which the frontend still expects a response.
I am open to any backend change (Tomcat / JVM or otherwise)
P.s. I understand that tomcat doesn't have any info about the accept queue. I read that the comm stack has that info. But what could be done is still unclear to me
If you don't want to wait after a certain period of time, you need to not queue the request in the first place. The only way to do that is to set the TCP backlog to zero and hope the OS cooperates.
You might want to re-think a couple of things, here, with your application.
First, you can adjust the TCP backlog, but of course requests can take an arbitrary amount of time to complete. So if you have just 1 request processing thread and it's busy, then the client will have to continually make requests until it can get a free thread (because the backlog is zero). The time for the in-flight request to complete is un-knowable, so you will have to re-try.
Second, the client can't know whether its request is waiting in the TCP backlog or if it's being actively served, so it either has to wait until a response is received or time-out. If you can't tell whether the request has started, you won't really know whether or not it's worth waiting.
Third, any TCP connection could potentially be interrupted or fail at any time. So if the server receives the request it may process it even if the client doesn't get a response. (This is essentially the same situation if the client has given up.)
There are a few ways to handle the above situations.
One way is to use a feature of HTTP called 100-Continue where the client makes a request by sending the request-line and headers including Expect: 100-continue header but no request body along with a POST or PUT request. The server won't reply with the 100 Continue response code until it's ready to process the request. If you don't get a 100 Continue within your e.g. 30-second window, you close the connection and then ... don't continue with the request. If you do get a 100 Continue response, you can then send your request body which, presumably, will include everything necessary to actually serve the request. If the server gets no request body, it can fail (somewhat) gracefully and the request is essentially ignored.
Another way for the server to detect whether the client is still there is to attempt to write data back to the client. Buffering in the server, OS, and network can sometimes make this impractical, and it is likely to degrade your network performance (because the response must be chunked in order to do this). If the client is gone and you aren't being hampered by the OS, then you'll get an IO error and your server-side process can abort.
Another thing you might want to do is ensure that all (or most) of your processes are idempotent. Basically, allow repeated requests for the same thing to not be a problem (other than for performance, of course). You can use things like transaction-tokens or some other "expirable" resource that allows clients to request the same thing over and over again but it will only succeed once. And, for your purposes, you want the checking for validity of such a token to be very fast so it doesn't cause a performance problem.
That's a lot to swallow, but hopefully there is something in there that's helpful to you.