I'm using the WriteKafaP
vertex in a Jet job. Is there a way to plug in custom partition logic when publishing to the Kafka topic?
In 0.4, you can use a custom Kafka partitioner configuration. In 0.5, we added the ability to map the item to a ProducerRecord
directly.
For example:
Vertex v = new Vertex("kafkaSink", KafkaProcessors.writeKafkaP(properties, o -> new ProducerRecord(topic, partition, key, value));