Search code examples
javaspring-bootspring-cloud-streamspring-cloud-dataflow

Spring Data Flow streams with multiple inputs


I use Spring Cloud Data Flow 2.11.2.

My goal is making two sources and one processor that accepts both messages as input and performs some operations on them, e.g.: stream view

I found a guide on this topic and the corresponding video. Also there're sources used in the guide.

However, when repeating actions from the video, SCDF does not display multiple inputs on the processor. Moreover, other answers to stackoferflow indicate that graphically SCDF does not support it.

In this case, I don’t understand how the author of the video managed to do this and what I have to do.

Thanks in advance!


Solution

  • It can be done w/ Dataflow but it is not as seamlessly supported as single input/outputs. You will have to use a slightly different DSL syntax and wire the components together yourself.

    Specifically, you have to register the application as type "App" and then use "||" rather than the "|" when creating the stream definition.

    You can find more details on this topic in the Dataflow Microsite: https://dataflow.spring.io/docs/concepts/architecture/#streams-with-multiple-inputs-and-outputs

    Also, Spring Cloud Stream itself supports multiple inputs/outputs: https://docs.spring.io/spring-cloud-stream/docs/current/reference/html/spring-cloud-stream.html#_functions_with_multiple_input_and_output_arguments