Search code examples
javatomcattomcat8server.xml

How to make Tomcat8 return 503 status code?


I have an application which process 1 request in ~100ms. What I intend to do is return 503(service unavailable) for requests when I'm getting more requests than I can handle. My server.xml for Connector component in tomcat8 looks like this

<Connector port="8080" protocol="HTTP/1.1"
                   connectionTimeout="200"
                   URIEncoding="UTF-8"
                   redirectPort="8443"
                   maxThreads="200"
                   acceptCount="0"
                   maxConnections="200"
/>

Going by the definition provided in the documentation, my acceptCount component is the queue when all my maxThreads are processing(being used). maxConnections is the number of connections server will accept or process at any given time. I'm intending on processing only 200 requests at a single moment. The load testing I'm doing is by apache benchmark with the following config. ab -l -n 1000 -c 1000 -s 100, the overall latency increases from 100ms to 2000+ms but still ALL my requests are being processed. Any idea what I am doing wrong?

EDIT1: As suggestied by dit I ran it with 20000 requests and now I'm receiving apr_socket_recv: Connection reset by peer (104) which as mentioned here suggests that it is due to config for the tomcat, so, thats a plus. However, Why is this error being sent and not an HTTP4XX or 503 response?

EDIT2: I'll be trying the method as mentioned in the answer I wrote.

EDIT3: So to obtain the information on how many Tomcat threads are busy and how many connections are currentyl established I am enabling JMX on Tomcat. I retrieve the mbeans info for tomcat


Solution

  • The link suggests that I'll have to myself monitor the number of threads working and If Im overloaded and send the response accordingly.

    Track the number of active concurrent requests in memory and use it for fast failing. If the number of concurrent requests is near the estimated active threads (8 in our example) then return an http status code of 503. This will prevent too many worker threads becoming busy because once the peak throughput is hit, any extra threads which become active will be doing a very light weight job of returning 503 and then be available for further processing.

    apr_socket_recv: Connection reset by peer (104) will be sent If I'm not handling the requests myself.