Handle parallel async operations with futures in Spring App

Question

I have a db with +1M entries in it, I am querying the DB in chunks (lets say chunkSize = 1000).

Iterating over this chunk of 1000, I want to publish a message to rabbitMQ for each entity, collect all the futures for this chunk into an array/list.

Wait on this futures array until all futures are completed/failed, then filter this array of 1000 into two new lists (successList, failureList), then prepare two bulkUpdate operations for mongoDB, and execute the two ops.

I have to wait chunk by chunk, because otherwise, the next iteration (for fetching db entities) will re-read already in progress entities.

PS: My initial assumption was to use CompletableFutures in a list (the same thing I would have done in Javascript, not sure if java has any other approach)

Answer 1

As explained in this answer Spring AMQP allows you to wait on a Future in CorrelationData to get the result of publishing a message. You don't need to invent your own mechanism.

You can always subclass CorrelationData if you want to store more information about a send.