How to send message to a particular partition in Kafka?

Question

I have created a topic that has many partitions. Using the console producer I want to send messages to particular partitions and view the through the console consumer. On the console producer I have tried this,

kafka-console-producer.bat --broker-list localhost:9092 --topic sample  --property parse.key=true --property key.separator=,

Send messages as,

key1,another-message

But I am just confused on whether key1 represents partition number.

Using the console consumer I viewed the messages,

kafka-console-consumer.bat --zookeeper localhost:2181 --topic sample

I want to view the messages according to the partitions. Is this the right way to view the messages on the console consumer? Can anyone please provide a clear understanding on this?

Answer 1

You can specify partition number directly in the ProducerRecord, but not with kafka-console-producer.

The key is not the partition number but Kafka uses the key to specify the target partition. The default strategy is to choose a partition based on a hash of the key or use round-robin algorithm if the key is null.

If you need a custom algorithm to map the messages to partitions, you need to implement org.apache.kafka.clients.producer.Partitioner interface. The name of you class must be set as a partitioner.class property of the producer.