How to concurrently fire multiple list of Objects independent of each list using Multi threading in Java

Question

I have a unique requirement for which i am unable to find relevant references online.

We usually get huge map of employee objects from a data warehouse application every weekend, where for every employee object in that map, we need to process a list of activity events. The catch here is, the events for each Employee object need to be processsed in sequence, and if some event failed to process, then we exit processing the complete event list for that Employee.

Like this, we need to concurrently process all the Emp objects in the map on their own threads.

For example, i will take 3 employee objects.

Emp1 -> E1Activity1,E1Activity2,E1Activity3
Emp2 -> E2Activity1,E2Activity2
Emp3 -> E3Activity1,E3Activity2,E3Activity3,E3Activity4

In this case, there should be 3 threads created, and each employee object, should start processing each activity on its own thread, and only if the one activity is completely processed, then it should move to processing next activity.

And, the number of Emp objects can change to any number, and not necessarily 3 as explained above.

Based on above, how can i create threads in such a way that i can keep firing requests in sequence, while also concurrently creating other threads which does the same.

Any help is appreciated. Thank you in advance.

Answer 1

You need one thread per employee. Within that thread, process that employee’s activities in proper order.

If “process a list of activity events” involves blocking, such as logging, reading/writing files, accessing a database, or making network calls, then your situation is a fit for the virtual threads feature in Java 21+.

Assign each of your incoming Employee objects as a task (Runnable/Callable) to an executor backed by virtual threads.

try(
    ExecutorService executorService = Executors.newVirtualThreadPerTaskExecutor() ;
) {
    for( Employee employee : employees)
    {
        EmployeeProcessingTask task = new EmployeeProcessingTask( employee ) ;
        executorService.submit( task ) ;
    }
}

In your EmployeeProcessingTask class, you process each of the Activity objects for that employee in the proper order. If any fail, end the looping through those Activity objects.

Each employee gets their own thread. So each employee’s activities are processed one at a time in order. All the employees are being processed concurrently, so some may finish before others.