Search code examples
apache-kafkaspring-kafkaspring-cloud-stream

spring kafka stream consume messages from multiple topics using functional style not working


I am trying to consume messages from multiple topics using single binding and just simply print. But for some reason I am only able to get the messages from the first topic. Do I have specify the property differently. I followed multiplex topic discussion from this blog https://spring.io/blog/2019/12/03/stream-processing-with-spring-cloud-stream-and-apache-kafka-streams-part-2-programming-model-continued.

    // My properties

    spring.cloud.stream.bindings.routeRequests-in-0.destination=kafk.pds.orch.be-uda.complete,kafk.pds.orch.prov-uda.complete,kafk.pds.orch.location-uda.complete
    spring.cloud.stream.bindings.routeRequests-in-0.consumer.use-native-decoding=false
    spring.cloud.stream.kafka.streams.binder.functions.routeRequests.applicationId=kafk.pds.orch.stream.routeRequests

// My code

@Bean
public Consumer<KStream<String, String>> routeRequests() {
    return uda -> uda.foreach((s, request) -> {
        System.out.println("Hello:" + s);
    });

Solution

  • I don't see any issues with your code. I just verified with a sample application that this feature works. See here.

    Compare this sample app with your application. If things still don't work, feel free to share a reproducible sample so that we can triage further.

    Btw - you don't need to set use-native-decoding to false, unless you have a specific reason to do so. The default is true (which means we rely on the Serde mechanism from Kafka Streams). However, this is not your issue. I was able to run the sample app with both settings.