Kafka Partition is null

Question

I am using spring kafka template to write messages to a kafka topic, and following API is used for publishing messages.

public ListenableFuture<SendResult<K, V>> send(String topic, K key, @Nullable V data) {... }

The good thing about using this API is that it returns a sendResult object which has some metadata like topic, partition, headers etc. I have a callback which prints the sendResult upon onSuccess. Now the issue is, the partition is being printed as null.

Instead of call back, I tried changing this to future.get(10, TimeUnit.SECONDS) to get the sendResult. Even then the partition is printed as null. Can anyone pls let me kow what could be the reason for this? Is this a default behaviour of kafka?

Answer 1

You probably need to take a look for that info in the SendResult.getRecordMetadata() instead of getProducerRecord().

The ProducerRecord is not modified by the KafkaProducer. So, the API you use is ended up like this:

public ListenableFuture<SendResult<K, V>> send(String topic, K key, @Nullable V data) {
    ProducerRecord<K, V> producerRecord = new ProducerRecord<>(topic, key, data);
    return doSend(producerRecord);
}

Where the partition property of the ProducerRecord is indeed null.

The RecordMetadata, on another hand, is indeed built already after the server answer. You can get a required info from its partition() option.