Search code examples
apache-kafkaspring-cloud

What is the difference between Apache Kafka and Kafka Streams on Spring Cloud Stream?


In the Spring Cloud website (https://spring.io/projects/spring-cloud-stream), are listed the binders options to use. And there we have the Apache Kafka and the Kafka Streams options.

  1. What's the difference between them?
  2. For what purpose we should choose between these two?

Solution

  • The Apache Kafka binder is used for basic kafka client usage consumer/producer api,

    Kafka Stream binder is built upon the base apache kafka binder and adds the ability to use kafka streams api,

    Kafka streams api is lightweight code libraries which gives you the functionality to manipulate data from topic/s in kafka to other topic/s in kafka , allow you to transform, enhance, filter,join, aggregate and more...


    The Apache Kafka Binder implementation maps each destination to an Apache Kafka topic. The consumer group maps directly to the same Apache Kafka concept. Partitioning also maps directly to Apache Kafka partitions as well.

    The binder currently uses the Apache Kafka kafka-clients version 2.3.1. This client can communicate with older brokers (see the Kafka documentation), but certain features may not be available. For example, with versions earlier than 0.11.x.x, native headers are not supported. Also, 0.11.x.x does not support the autoAddPartitions property

    https://docs.spring.io/spring-cloud-stream-binder-kafka/docs/3.1.3/reference/html/spring-cloud-stream-binder-kafka.html#_apache_kafka_binder

    Spring Cloud Stream includes a binder implementation designed explicitly for Apache Kafka Streams binding. With this native integration, a Spring Cloud Stream "processor" application can directly use the Apache Kafka Streams APIs in the core business logic.

    Kafka Streams binder implementation builds on the foundations provided by the Spring for Apache Kafka project.

    Kafka Streams binder provides binding capabilities for the three major types in Kafka Streams - KStream, KTable and GlobalKTable.

    Kafka Streams applications typically follow a model in which the records are read from an inbound topic, apply business logic, and then write the transformed records to an outbound topic. Alternatively, a Processor application with no outbound destination can be defined as well.

    https://docs.spring.io/spring-cloud-stream-binder-kafka/docs/3.1.3/reference/html/spring-cloud-stream-binder-kafka.html#_kafka_streams_binder