Search code examples
javaapache-kafkaspring-cloud-stream

Spring-cloud-stream-kafka -- Getting frequent "Attempt to heartbeat failed since group is rebalancing"

I have a spring boot application, running with 25 instances, have 25 partitions. It consume the message and do some calculation by getting the data from cache and put the resultant value into database.

But the performance is very poor for that app, even with these many instances, CPU usages are very high. Max time taken per transaction is 6-8s, average time --> 700ms

In Splunk, IN A MINUTE, I see this message: [Consumer clientId=consumer-2, groupId=TEST] Attempt to heartbeat failed since group is rebalancing thread_name: kafka-coordinator-heartbeat-thread | CG.LOGICALINVENTORY-TEST around 200 times.

My question is, Why do I get this message frequently?? and can this(getting frequent rebalance) cause the poor performance.

Any thoughts??

Solution

  • Increase max.poll.interval and/or decrease max.poll.records (Kafka consumer properties).