Search code examples
apache-kafkaavroapache-kafka-connectconfluent-schema-registryksqldb

ksqlDB generates "connect.name" element in AVRO schema and immediately kicks it out again in new version


I'm defining a new stream with a new topic via Confluent Cloud ksqlDB GUI, automatically registering a new schema with no previous versions.

v1 of the schema looks as expected, but starts with an element "connect.name": "my.namespace", which I already don't really understand. However, there is also immediately a new version of the schema generated, which is v2 and lacks this element. How can this behaviour be explained? There are no connectors in place (which is usually the context I would expect from a connect element) and this has been the case for all ksqldDB-related schemas as far as I can see.

Furthermore, I could observe that doc fields are evolved "out" of existing AVRO schemas (in the context of ksqlDB-related schemas), which adds up to behaviour that does not meet expectations, maybe it's related because CC schema registry kicks out non-required fields by default?


Solution

  • If there are no connectors running, then you are correct that connect.name should probably not be generated.

    However, ksqlDB may use Connect Struct types and associated AvroData class methods to/fromConnectSchema to convert between Kafka records and internal ksqlDB Row datatypes that can be queried, and these are used by AvroConverter in Connect framework to read/write to external systems, and what are generates and registers the schema you're seeing.