Search code examples
ksqldb

KSQLDB create a stream from multiple streams without join


is it possible to join multiple streams to one stream without a join / window clause? I just want something similiar as a combined kafka topic, where all messages can be found for further processing.

                     /--->transformationOnValueX stream ---\
kafka topic source1:/---->transformationOnValueY stream ----\
                     /--->TransformationOnValueW stream -----\
kafka topic source2:/---->TransformationOnValueZ stream ------\------> combind_stream_all_messages_transformed_multiple_sources

Any idea how to do that?

Edit:// Found https://kafka-tutorials.confluent.io/merge-many-streams-into-one-stream/ksql.html but that solution requires an insert into statement manually after the novel stream is created. Is there a way to do it in one statement?

As a side effect the event driven nature is broken, or am i wrong? what happened if an event is published to the source topic after the insert into statement? is it lost?

Edit:// To inform all fellows. If you use a insert into select... a running query will be created and the final stream will receive further updates


Solution

  • To inform all fellows. If you use a insert into select...-statement, a running query will be created and the final stream will receive further updates.