Search code examples
apache-kafkaksqldb

Changing the source topic name using a CREATE STREAM query in ksqldb headless deployment mode

I have a topic whose name contains dots, it was created with the Debezium connector, it's there and I can't do much about it.

With ksqldb I must create a materialized stream view out of it, but topic names with dots are not allowed in the SELECT clause.

So I must first declare the original topic with a new name, right?

CREATE STREAM datachanges_my_event WITH (KAFKA_TOPIC='datachanges.my.event', <...>);

and then create the materialized stream view

CREATE STREAM my_filtered_event AS SELECT <expr> FROM datachanges_my_event WHERE <condition> EMIT CHANGES;

It's all good, but now I must try my best to do this as part of a deployment in headless mode; however, the first ksql query is not persistent and does not make sense in headless mode. On the other side, I cannot run the first query in interactive mode and the the second one as part of a headless deployment mode.

So, please, how can change the name of a source topic within a query run as part of a ksqldb headless deployment?

Thank you.

Solution

  • One cannot run a CREATE STREAM query in headless mode, without having at least also an CREATE STREAM AS query to create a materialized stream view.