Scalable spring scheduler for mongodb fetch

Question

I have mongodb collection named A. In that collection there is date and time field named field1. In my scenario I have to collect records which has expired date and time based on field1.

I thought of writting spring scheduler Job which runs on every min to fetch date and time expired records from mongodb but when I start writing I could not make spring scheduler as scalable.

In scheduler, convert the record to String and do a message push to Kafka topic. In my case spring scheduler runs on 4 instances and push duplicate messages to Kafka topic.

Question is, In spring scheduler, how can I stop duplicate messages to Kafka topic or how to achieve scalable scheduler with out any duplicate messages?

Any event based framework to support my scenario? Means when the record expired it should immediately notify the java system, ie. Similar to TailableCursorlisteners in spring listens to cappedcollection.

Answer 1

You can use a shedlock library which will ensure that only one job through all instances is running Here is tutorial for more information:

https://www.baeldung.com/shedlock-spring