Search code examples
javaspringtomcatconcurrencyspring-integration

Move logic from client to server with preserving concurrency (Java)

What I have:

  1. There are 10 requests that are sent from the client app to the server to get information about delivery.
  2. After responses for all 10 requests are received logic is executed on the client to calculate the best delivery.
  3. Best delivery has shown to the user.

What I want:

  1. One request is sent from the client to the server.
  2. Server splits it into several tasks and processes in parallel (because tasks may require computation and call of other integrations via HTTP).
  3. Server computes all results and calculates the best delivery.
  4. Response returned with the best delivery to the Client.

Idea - moving logic from the client to the server because API may be used by different systems (web, mobile, etc.)

Problem - if I will use a thread pool, I'm afraid that there may be thread starvation. Currently, concurrency is managed by the server itself (Tomcat, I believe also uses a thread pool but it's configured and optimized as far as I understand), and there are no issues. But when I move it to java code I'm afraid there will be problems.

Tools - Java 8, Spring 4(no spring-boot), Spring integration.

My idea is to use Executors.newCachedThreadPool(); but I'm not entirely sure about this solution, because it may grow out of control. Could you provide options for this change?

Solution

  • What you describing fully fits into a Scatter-Gather pattern: https://www.enterpriseintegrationpatterns.com/patterns/messaging/BroadcastAggregate.html

    See its implementation in Spring Integration: https://docs.spring.io/spring-integration/docs/current/reference/html/message-routing.html#scatter-gather

    What you need is a PublishSubcribeChannel with an injected ThreadPoolTaskExecutor to be sure in a sufficient number of concurrent executions.

    The Tomcat by itself is also a Java process and takes resources from the environment. Therefore no matter how it is configured you really may end-up with some peak where number of thread is not enough to serve your users. From here you would go into a cluster and have a load balancer upfront. But that's fully different story. The answer to your question is a Scatter-Gather with an async PublishSubcribeChannel for an Auction scenario.