I'm maintaining an application which has few services exposed as SOAP webservices. I have come across some scaling issues off late, as application has been receiving more load than the normal.
I would like to know if my understanding is correct regarding few pool configurations, 1. We have the thread pool configuration as below,
<thread-pools>
<thread-pool name="admin-thread-pool" max-thread-pool-size="50" max-queue-size="1024"></thread-pool>
<thread-pool name="http-thread-pool" max-thread-pool-size="250"></thread-pool>
<thread-pool name="http-thread-pool-internal" max-thread-pool-size="50"></thread-pool>
<thread-pool name="thread-pool-1" max-thread-pool-size="200"></thread-pool>
</thread-pools>
and
<transports>
<transport name="tcp" acceptor-threads="8"></transport>
</transports>
and 2. EJB pool configuration as below,
<ejb-container steady-pool-size="0" max-pool-size="50" pool-resize-quantity="10">
So now the questions.
What will happen if http thread pool receives a task which need to be executed synchronously and if there are no enough EJB bean instances in pool (maximum configured is 50), because all EJB instances are serving other http requests ? Note : We are doing JNDI lookups and not using @EJB annotations.
Does it make sense to increase the number of EJB instances (max) equal to http-threadpool value ?
After doing some profiling it was noticed that the code which does lookup for the EJB instances take a long time to return. Does it mean that there was no EJB instances available and the request had to wait until an instance was release by the other running threads ?
What will happen if http thread pool receives a task which need to be executed synchronously and if there are no enough EJB bean instances in pool (maximum configured is 50), because all EJB instances are serving other http requests ?
EJB pooling is not part of JEE specification, therefore, pooling behavior is vendor dependent. According to Glassfish documentation (may be this can vary between versions) if the pool size is less than steady-pool-size, when a new request arrives the Container will create X new ejb instances where X is determined by the pool-resize-quantity value. Therefore, you shouldn't run out of pooled ejb. The max-pool-size is not a fixed limit.
Does it make sense to increase the number of EJB instances (max) equal to http-threadpool value ?
It could be right but notice that the Container will increase automaticly the pool size if more ejb are needed during a high-load period.
I would change the steady-pool-size to a value grater than 0 instead.
After doing some profiling it was noticed that the code which does lookup for the EJB instances take a long time to return. Does it mean that there was no EJB instances available and the request had to wait until an instance was release by the other running threads ?
In case (may be a wrong configured pool) the server runs out of available ejb, has more sence that it creates a new instance instead of to serialize clients requests. This what the specification(ejb 3.1) says about:
4.7 Stateless Session Beans ...The container creates another stateless session bean instance if one is needed to handle an increase in client work load...
Create a new ejb instance shouldn't be too expensive unless your bean has to execute specific business logic at the moment of initialization (e.g. logic at @PostConstruct annotated method).
Take in mind that under high load there are others more relevant issues than pool configuration that need to be monitored such as cpu and memory server usage.