I am wondering if there is a way to delay processing of events in a kafka stream based on the timestamp and some event data. for instance, say there are three events in the stream: event1, event2 and event3. all were produced to the stream at the same time and based on some data in the event I determine I need to process event1 after 10 seconds has passed in the timestamp, event2 in 60 seconds and event3 in 15 seconds. Is there a way to achieve this behavior without pausing the consumer? so after 10, seconds I could process event1, after 15 seconds process event3 and then after 60 seconds process event2. I have seen some answers about pausing but it seems that it would pause for 10 seconds, process event1, pause 60 seconds, process event2, etc. Any input would be greatly appreciated!
I believe it wouldn't be possible without pausing the container.
We did have similar use case, however, we wanted to wait till specified time and then start processing the messages. We ended up implementing the same using pause/resume feature.
Having a BackOff might be helpful, but the varying wait time (10, 60, 15) would make things difficult.
I am wondering whether storing data in DB view and a scheduler would be a good choice for your case.