Search code examples
javamultithreadingexecutorservice

Java: Executor Service with multiple queues


Requirement:

  1. I have messages grouped into different types e.g Type1, Type2 ... Type100.
  2. I want to execute different types of messages in parallel. Let's say in 10 threads, but all the messages of a same type must execute one by one. Execution order does not matter.
  3. Once a thread finish all the messages of TypeX. It should start processing another Type.

I went through the different answers: Most of them suggests executor service to handle multi-threading. Let's say we create executor service like

ExecutorService executorService = Executors.newFixedThreadPool(10);

but once we submit the message using executorService.submit(runnableMessage);

We don't get any control over the assignment of specific Type of message to a particular thread only.

Solution:

creating an array of single threaded executors

ExecutorService[] pools = new ExecutorService[10];

and initially pass the messages of Type1, Type2 ... Type10 then if any executor has finished execution then assign Type11 to it and keep doing it until all Types gets processed.

Is there any better way to do it?

Something like executor service with multiple queues where I can push messages of each type to a different queue?


Solution

  • A simpler solution could be:

    Instead of making each message runnable. We can create group messages according to their type:

    e.g. we create Group1 for all the messages of type1

    class MessageGroup implements Runnable {
        String type;
        String List<Message> messageList;
    
        @Override
        public void run() {
          for(Message message : MessageList) {
             message.process();
          }
        }
    } 
    

    And we can create usual executor service with fixed threads like

    ExecutorService executorService = Executors.newFixedThreadPool(10); 
    

    And instead of submitting individual messages we can submit the group of messages like

    executorService.submit(runnableGroup);
    

    and each group will execute the messages of same type sequentially in the same thread.