A question came up at work today:
Imagine that I have a lot of concurrent users accessing my website, each with their own data stored in user sessions. Considering that I have a limited amount of memory available in my JVM, what happens, or what is supposed to happen, when the concurrent active sessions reach a point where the container JVM runs out of memory?
I tried to find something in the servlet spec about what should happen but there is nothing definitive in there, it just says that the developer will have access to the session objects, etc. This makes me think that it must be provider specific.
Taking Tomcat as an example, does the webserver just start throwing OutOfMemoryExceptions? Or is it more clever about the way it deals with this, for example paging the sessions off to another cache? Or another way all together?
The servlet specification does not say anything about memory considerations, so you'll get no help there.
Unless configured specifically to do otherwise, Tomcat will allow you to use-up all available memory with your HttpSession objects (realy, their contents) and ultimately the JVM will start throwing OutOfMemoryErrors, potentially taking your server down (though the JVM will continue to run, many things will behave ... unpredictably and unfortunately).
If a single request starts using a lot of memory in local variables, etc., the request-processing thread will suffer an OutOfMemoryError and stop processing the current request. (I believe in this case, the request-processing thread will actually be recycled by the Tomcat request-processing thread pool). The garbage collector will likely run shortly thereafter and re-claim the memory used by that request and your server will stabilize.
On the other hand, if you use-up a lot of memory and store those objects into the user's HttpSession, then the GC cannot free any memory and your server will continuously suffer from OutOfMemoryErrors. Though Tomcat will expire sessions on a specified schedule (the default is after 30 minutes of inactivity), the session-cleansing thread may encounter an OutOfMemoryError during operation and therefore fail to perform its duty, compounding the whole situation (because, effectively, HttpSessions will never expire).
There are several ways to mitigate the above unfortunate scenario. Which one makes sense to you is up to your requirements and environment.
Increase your heap size. This will obviously only get you so far. If you are filling-up a 16GiB heap with HttpSession and related objects, then you are reaching the limits of commodity hardware and you can't simply buy-a-bigger-box.
Reduce the session expiration time (default is 30 minutes). If a session isn't explicitly terminated (say, through a logout function), then the HttpSession object and all its contents stick around until the expiration interval has passed. If you have lots of data in lots of abandoned sessions, reducing the session expiration time may give you some breathing room.
Stop putting so much data into the user's HttpSession. While this may seem like an obnoxious suggestion ("stop doing that"), it is honestly a valid one: do you really need to store so much stuff in the session itself? How about using a data store of some kind (relational database, non-relational database like Cassandra, webcache, etc.)? Maybe store the data in a file on the disk? While these suggestions will certainly limit your ability to access your data quickly, they are certainly better options than having your server come crashing down under the weight of so much stuff in your session.
Use Tomcat's PersistentManager, which is a session manager capable of swapping active (but idle) sessions to some external storage (file-based and JDBC-based storage mechanisms are available by default). This can get you a long way while you figure out some other place to put all that data you are shoving into the user's session.
All but #4 are applicable to any servlet container. #4 should be available to JBoss users, as JBoss uses Tomcat internally. For other containers, you may find that a similar capability exists.
UPDATE 2025-01-29
I can't be bothered to check the state of things way back in 2012 when this answer was first written, but there are some other things to consider these days.
First, Tomcat currently has a default max-session-count infinite but it can (and probably should) be changed to something reasonable - where your application environment needs to define "reasonable". You can set the maximum number of sessions Tomcat will create for your application in the <Manager> attribute maxActiveSessions. This element goes into your application's META-INF/context.xml file which is a Tomcat-specific config file.
Second, there are some tricks you can use to tolerate memory pressure on sessions containing large amounts of data. Assuming that the large, in-memory stuff is not critically important (e.g. it's backed-up by some persistent storage such as a database, etc.) then you can simply drop it from memory. Since you can't really predict when memory might suddenly run out, your application can't proactively remove those objects. Instead, you can use things such as WeakReference object wrappers around your session values. Your application will have to un-wrap them and check for nulls, but it will allow the GC to automatically drop unnecessary objects from your sessions when memory pressure builds.
The suggestion to use PersistentManager above could be seen as a placeholder for any similar technique, such as using memcached, etc. for storage of your sessions. Just be aware that moving to memcached specifically just moves your problem to another chunk of memory which is, by definition, limited.
I think if you have huge sessions, the goal should be to have smaller sessions. If you can't make them smaller, consider off-loading that data to another session-store (db, etc.). If you have a large number of sessions, then you should consider horizontal scaling of your application servers (e.g. Tomcats).