Search code examples
apache-kafkamicroservicesspring-cloud-stream

Reactive kafka stream example with spring boot webflux

I am trying to build a Spring Boot WebFlux application which consume events from one Kafka topic does some processing(would involve operation of reactive db lookup) on it and send them to another kafka topic. Currently the application uses webflux and spring cloud data streams for kafka.

Can some please suggest is it fine to use Spring Cloud Stream Kafka Stream API or should I use something else to achieve end to end reactive behaviour.

Tried with Spring boot WebFlux, spring cloud data stream kafka streams.

Processor example here

enter image description here

Solution

  • It does not look like you are using the Spring Cloud Stream Kafka Streams binder, but you mention Spring Cloud Stream Kafka Stream API . I assume you are using the regular Kafka binder in Spring Cloud Stream based on message channels since nothing in the code you provided points to any Kafka Streams API. Assuming this is the case, the best way to achieve full end-to-end reactive behavior is to use the new reactive Kafka binder (not yet released, but part of the 4.0.0 release, which is presently in the RC stage). More details on this reactive Kafka binder is available here. This binder implementation is based on the reactor Kafka project from Project Reactor. The regular message channel-based Kafka binder is not based on reactive models and could block when consuming/producing. Therefore, if you are looking for an end-to-end reactive paradigm, use the reactive Kafka binder.