Process multiple request at same time

Question

I have a service class

    @Path("/")
     public class ABC {
      @Path(/process/{param})
      public String processRequest(@PathParam("param") String param){
        Thread t = new Thread(()->{
         // Do some processing with the param parmeter
          System.out.println("Processing started for -> "+ param);
          //Do many more things
          //DB transactions,etc.
           });
          t.start();
          return "Your request will be processed";
        }
}

I accept some parameter and start processing it in a new thread and at the same time so that it should complete the processing within 30 secs, I break my connection with the client by acknowledging him that his request will be processed.

It works fine and till now without any issues, Currently, it can process more than 5k requests. The problem starts when there is a lot of requests come at the same time maybe more than 50k so my application creates a new thread for every new request which causes the application to allocate a lot of memory and also sometimes makes JVM memory to exhaust.

Is there any another way by which I can immediately start the processing without bothering for the number of requests and process all the requests within 30 secs and also limit the no of threads active working threads.

One way I found was the Producer-Consumer implementation in which I can accept all the requests and put simultaneously into the producers and my consumers pick up the request and start processing it, For this implementation i need to specify the maximum no of request which can be accepted by producer(Ex : 100 000) and no of consumers which can process the request(Ex : 1000) so that only 1000 threads are active and process one after another but issue with this approach is that if any of the consumer (working) thread if locks due to some reason and if not released then there are only the remaining unlocked threads left to process the request and the incoming request is continuously increasing in the producers. Only increasing the no of consumers creates more working thread but at the same time there can be a lot of locked threads processing the task.

Please let me know any another approach by which I can do it.

Note : All the request should be processed within 30 secs and if unable to do then it fails the success criteria.

Answer 1

You probably want a queueing mechanism, like RabbitMq.

Your application will run like this:

request -> push to queue -> return ACK to client

queue -> worker threads.

The queue consumer speed is determined by your worker threads speed, so you will never exhausted your system.

Under load, there're lots of message will be queued, mean while your workers reliably takes messages from queue and process them.