Search code examples
avroapache-kafka-streamsconfluent-schema-registry

kafka streams joinWindow and auto create a avrò schema


When I use kafka stream joined windows, auto create an avro schema

like this " * KSTREAM-JOINTHIS-0000000125-store-changelog-value"**

I want to know, why this can create avro schema ?

there is my code:

Serde<FactCallProviderMessage> specificAvroSerdeForCallProviderMessage = ProcessStreamUtil.getAndRegisterSerde(isKeySerde);
        KStream<String, FactCallProviderMessage> callProviderMessageKStream = builder.stream(
                callProviderMessageTopic /* input topic */,
                Consumed.with(Serdes.String(), specificAvroSerdeForCallProviderMessage));


public static <T extends SpecificRecord> Serde<T> getAndRegisterSerde(boolean isKeySerde) {
        Serde<T> specificAvroSerde = new SpecificAvroSerde<T>();
        specificAvroSerde.configure(Collections.singletonMap(
                AbstractKafkaAvroSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG,
                MyConfig.getSchemaRegistryUrl()),
                isKeySerde);
        return specificAvroSerde;
    }


Solution

  • Kafka Stream creates so-called changelog topics for stateful operators like join in the background to backup the state in a fault-tolerant manner in the Kafka cluster.

    If you use Avro format for input messages, your input messages will be written in Avro format to this changelog topic. Thus, on write the corresponding schema will be registered for this changelog topic.