Search code examples
javatomcatapache2timeoutanalytics

Tomcat configuration for analytics application


I am working on an analytics application very similar to Google Analytics. It basically stores the statistics from various sites in a database.

The application has the following properties:

  • large number of requests (2.5k/s at peak time)
  • little processing per request

I am using Tomcat 6 and the Apache HTTPD 2.

What values I should set for Tomcat's maxThreads and connectionTimeout attributes for this particular type of application?


Solution

  • The only way to determine what the optimal settings are for your particular site would be through testing. Clearly your Tomcat instance will be handling a very large number of fairly small requests with little returned data - not a normal situation. My guess is that you would want a high number of threads (1000 maybe, default is 200) and a short timeout but what exact values would be optimal would have to be determined by testing and possibly tweaked after field trials.

    In the past I've written little multi-threaded applications to fire requests at a server, it's cheap and cheerful and can get to very high request rates. If you want something more sophisticated check out JMeter it can load test most sites pretty well.

    I think you should be considering running multiple Tomcat instances with some form of load balancing if you are worried about the number of requests swamping an individual instance. Since you seem to be running Tomcat behind Apache that should be fairly simple to set up in the AJP connector.

    EDIT

    Just saw your update with figures. I prefer to look at requests / second and the figures you give work out at about 18 / second. I would have thought Tomcat would be able to cope with that without changing the settings. My guess is that you have a bottle neck somewhere else, probably in the storage of the data as I'm guessing it's going into a database.