Is there a way to read only X number of message in particular time period suppose 1 minutes using Akka kafka Stream Consumer https://doc.akka.io/docs/alpakka-kafka/0.15/consumer.html via some configuration. Need to handle a situation where there is bombardment of messages from producer at particular time so consumer can be impacted.
The throttle stage in Akka Streams can be used to limit the rate at which elements are passed to the next stage in a stream (using the Scala API):
.throttle(100, 1.minute)
This will still process every message in the barrage, but if there's enough time between bursts processing can catch up.
Note that Akka Streams has a robust backpressure mechanism: downstream stages will only demand enough messages that they can handle. This in effect makes processing self-throttling: the throttle stage is useful if that's not enough or you've otherwise "opted out" of backpressure.
If there's not enough time between bursts for processing to catch up, a buffer stage can be put in front of the throttle with a drop-message-if-full strategy. This will prevent backpressure from the throttle from propagating to the Kafka consumer, e.g.:
.buffer(1000, OverflowStrategy.dropHead) // Drop the oldest message
.throttle(100, 1.minute)