Search code examples
javaapache-kafkakafka-producer-apikafka-topic

Get the partition number of a record before producing in Kafka

I need to know the partition number of Kafka topic to which a Kafka record goes.

Before the execution of

producer.send(record);

Is there any way to know to which partition that record goes?

Solution

  • AFAIK it's not possible using the default round-robin partitioner. If you specify a key you could get the default algorithm from the Producer source code and try to predict (it's kind of hash(key) % num.partitions) If you use a custom partitioner you process it and already know. I was wondering about the ProducerInterceptor but it provides information before the partition is assigned as you can see from doc.

    https://kafka.apache.org/26/javadoc/org/apache/kafka/clients/producer/ProducerInterceptor.html