Search code examples
apache-kafkakafka-producer-apiapache-kafka-connect

Kafka Connect writes data to non-existing topic


Does Kafka Connect creates the topic on the fly if it doesn't exist (but provided as a destination) or fails to copy messages to it?

I need to create such topics on the fly or programmatically (Java API) at least, not manually using scripts.

I searched this info, but it seems topics have to be already created before migration


Solution

  • Kafka Connect doesn't really control this.

    There's a setting in Kafka that enables/disables automatic topic creation.

    If this is turned on - Kafka Connect will create its' own topics, if not - you have to create them yourselves.