I've got a Spring Boot web application (with Tomcat), serving HTTP APIs. There are multiple clients, each has its own set of APIs (for example Foo client can only calls /api/foo/** APIs).
To isolate each client from others, I'd like to have dedicated thread pool for each client, so that for example, in case of a sudden spike in the API calls from Foo client, other clients don't get throttled.
I searched a lot for a solution, but couldn't find a clean one.
For example one solution was to create a This is not gonna work, at least based on what I see in Filter with one Executor per client, and dispatch each request to the corresponding Executor. But I'm not sure if it's a good idea to do this.CoyoteAdapter.
I also thought about writing a custom delegating Executor for the Tomcat Connector which has several child Executors and decides which one should be executing the API call. But I'm not sure if it's even possible to do that.
The only clean solution I found is multi-connector setup which makes the app to listen to multiple ports and I'd like to avoid the additional complexity if possible, since it leaks to the infrastructure level.
I saw a similar question here, but slightly different, and was hoping there's a solution for mine.
What's the best way to do this? Is there an easy clean native way to do this?
Thanks.
It turns out be very easy actually. I can utilize Servlet Asynchronous Processing feature, simplified by Spring:
@GetMapping("/foo")
@ResponseBody
public DeferredResult<String> foo() {
DeferredResult<String> deferredResult = new DeferredResult<>();
// Save the deferredResult somewhere..
return deferredResult;
}
// From some other thread (e.g. a method annotated with @Async)
deferredResult.setResult(result);
This way I get to choose where to run the actual logic (in a different thread from a different executor). There's some nuances like context propagation, but totally doable.
More details here: https://docs.spring.io/spring-framework/reference/web/webmvc/mvc-ann-async.html