Search code examples
javaajaxmultithreadingwebspherethreadpool

How to handle high thread usage because of multiple AJAX requests in WebSphere

I've been told during stress testing that I need to reduce my app's Thread usage.

All of our other apps follow a 1 request per page model while I've create a single page app that fires 8 AJAX requests on load.

I'm running on Websphere 8.5.5.1 (Servlet 3.0).

I am making 8 AJAX requests from the Browser one straight after the other to the same Servlet, so they are in Parallel.

The Servlet basically checks the request parameters and calls the corresponding ESB service over http and then returns the response.

Please correct me if anything I say here is rubbish as I'm not 100% on any of it...

My understanding is that when the request hits Websphere it gets a thread from the thread pool and uses that thread to execute the Servlet and holds on to the same thread until the response is returned. I'm also thinking that the thread will block while waiting for a response from the ESB. Is that right?

I'm looking at the following 3 options:

1) The best we could do with Async servlets in Servlet 3.0 is put the request on a queue and return the thread back to the pool. Then one or more threads would go through the queue servicing the requests. So we are just swapping one thread pool for another, but we can limit/control the size of the second one.

If we had Servlet 3.1 (JSR 340) we could use a ReadListener and I'm guessing we could avoid the thread blocking while waiting for the ESB response?

2) Queue the AJAX requets on the Frontend so instead of firing off 8 AJAX requests, I fire 3 and when they finish fire the next one and so on, so you never have more that 3 requests in parallel.

3) Make 1 AJAX Request to the Servlet and let it call all 8 ESB Services and combine the results.

Can you advice me if any of the above solutions are valid and if I even understand them fully and suggest other possible solutions?

Thanks.

Solution

  • Trying to answer point by point:

    1. "Websphere it gets a thread from the thread pool and uses that thread to execute the Servlet and holds on to the same thread until the response is returned." Your understanding is incorrect. This would have been correct if WS was using old IO, but WS version 6.1 onwards use Native IO (NIO) and Asynchronous IO (AIO), which allows a handful of threads to cater thousands of concurrent connections. See this reference. Thus, you should not worry about making concurrent connections from AJAX client, and parallelism is fine.
    2. Having said this, the application must try and avoid doing any blocking operations, that block any worker threads which otherwise are capable of handling multiple concurrent connections. Just think on how you can execute each task async, which would mean that you should be able to do things without blocking yourself. In your case, you are waiting (and blocked) on the responses from ESB. There will be ~8 blocking connections to ESB service. Please review how you are making calls to ESB, and if you can use NIO for these calls. Understand that WS is a client for calls to ESB, and it has to use non-blocking IO for the operations it does on ESB.
    3. Alternatively: Is there a way, that ESB can also behave async, and release the WS request thread? ESB can callback a WS service URL, when its done with the request, and you can then create the AJAX response. That way you would not hold up WS threads.

    In your options above, clearly #1, and #3 will not help you. #2 would of course limit the parallel operations to ESB to three. But, I would recommend to see if the calls to ESB can be done in a async manner, and non-blocking manner. That would be ideal.

    Update

    To make an async and non-blocking calls to an external service from your app server, consider using a library like async-http-client. This uses Netty by default, but other NIO frameworks can be plugged in as well. The README.md of this project has good examples on how to make an HTTP call, and write response on the completion of HTTP request. You can use the onCompletion to write response back to your client.

    The concluding statement:

    1. Be a non-blocking server when listening to your clients. So far as possible, make sure that the request threads of the container/server do not block.
    2. Be a non-blocking client when making calls to an external service. This will allow you to make a large number of requests with a handful of threads.