Spring-Kafka with Micrometer throws NullPointerException during app shutdown

Question

We have spring-boot app with multiple nodes, and use KafkaTemplate for producing kafka messages. On classpath we have both org.springframework.kafka:spring-kafka and io.micrometer:micrometer-core. In our case, messages produced continuously, around a few thousand per second per topic (topics with multiple partitions). During app graceful shutdown (either during redeployment or automatic downscaling) we frequently have NPE errors:

[kafka-producer-network-thread | producer-1] ERROR org.apache.kafka.clients.producer.internals.ProducerBatch - Error executing user-provided callback on message for topic-partition 'xxx-1'
java.lang.NullPointerException
    at io.micrometer.core.instrument.Timer$Sample.stop(Timer.java:280)
    at org.springframework.kafka.support.micrometer.MicrometerHolder.success(MicrometerHolder.java:93)
    at org.springframework.kafka.core.KafkaTemplate.lambda$buildCallback$4(KafkaTemplate.java:587)
    at org.springframework.kafka.core.DefaultKafkaProducerFactory$CloseSafeProducer$1.onCompletion(DefaultKafkaProducerFactory.java:764)
    at org.apache.kafka.clients.producer.KafkaProducer$InterceptorCallback.onCompletion(KafkaProducer.java:1348)
    at org.apache.kafka.clients.producer.internals.ProducerBatch.completeFutureAndFireCallbacks(ProducerBatch.java:227)
    at org.apache.kafka.clients.producer.internals.ProducerBatch.done(ProducerBatch.java:196)
    at org.apache.kafka.clients.producer.internals.Sender.completeBatch(Sender.java:707)
    at org.apache.kafka.clients.producer.internals.Sender.completeBatch(Sender.java:688)
    at org.apache.kafka.clients.producer.internals.Sender.handleProduceResponse(Sender.java:596)
    at org.apache.kafka.clients.producer.internals.Sender.access$100(Sender.java:74)
    at org.apache.kafka.clients.producer.internals.Sender$1.onComplete(Sender.java:798)
    at org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:109)
    at org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:569)
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:561)
    at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:335)
    at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:244)
    at java.base/java.lang.Thread.run(Thread.java:834)

I checked implementation of class org.springframework.kafka.support.micrometer.MicrometerHolder, and see, that it happens only if destroy method invoked earlier than success. So it seems here we have a race condition. I guess success method implementation should somehow handle cases when destroy method already invoked. Is it a bug of spring-kafka?

spring-kafka version 2.5.1; micrometer-core version 1.5.1.

Answer 1

if was a bug Spring-Kafka with Micrometer throws NullPointerException during app shutdown #1531 in version 2.5.1. since spring-kafka version 2.6.0 issue is fixed.

meanwhile, if you don't need to have micrometer metrics for KafkaTemplate, we could disable it:

kafkaTemplate.setMicrometerEnabled(false);