Search code examples
apache-kafkaspring-integrationspring-kafka

What is the difference between producing message to kafka topic with Spring Integration MessageChannel and KafkaProducer?


I see there are two approaches to produce message to kafka topic, one with KafkaProducer.send(ProducerRecord<K, V> record) and other with(springframework.message) MessageChannel.send(Message). Former returns Future and latter returns boolean. I want to understand the difference between these two or advantage of one over the other, so I can choose the suitable approach for a particular case. I'd appreciate if anyone can explain on this.


Solution

  • The KafkaProducer is a plain Apache Kafka client API. You probably can do with it whatever you need writing all the logic yourself. That's fully OK.

    On the other hand if you would like to use more high-level API and your application is fully based on Spring, then you definitely should think about using Spring for Apache Kafka framework: https://spring.io/projects/spring-kafka. It provides a well-know template pattern implementation over the mentioned KafkaProducer - KafkaTemplate.

    Spring Integration with its Channel Adapter implementations for Kafka(https://docs.spring.io/spring-integration/docs/current/reference/html/kafka.html#kafka) comes handy when your application is based on Enterprise Integration Patterns and the logic is composed as integration flows. This way, when message is placed to the particular channel, a channel adapter subscriber is going to call KafkaTemplate for a KafkaProducer.send() underneath. And yes, this channel adapter implementation handles Future for us properly.

    You probably just need to learn more about Spring Integration and when to use it over the plain Apache Kafka API...