Kafka schema registry 409s on google.protobuf.struct.proto

Question

I'm building a Kafka Streams application using Protobuf for message schemas. For now the application itself is just piping from one topic to another. I'm running Kafka locally using the Confluent platform all-in-one docker-compose file.

One of my schemas (foo.proto) uses a Struct field, so prior to starting my app I have registered both foo.proto and struct.proto on the schema registry.

When I start my app the protobuf serializer runs a method called resolveDependencies, leading it to re-register subtruct.proto. The (local) schema registry returns a 409 with message:

Schema being registered is incompatible with an earlier schema

The code that originally registers struct.proto and the app are both using version 3.12 of the protobuf-java library. Looking at the docker logs for the registry, they give me a bit more detail with the following message:

Found incompatible change: Difference{fullPath='#/ListValue/1', type=FIELD_NAMED_TYPE_CHANGED}

Looking at that particular field the already registered schema has the following definition

message ListValue {
  repeated Value values = 1;
}

and the schema my application is trying to register has

message ListValue {
  repeated .google.protobuf.Value values = 1;
}

My questions

1. Why is the kafka-streams application trying to re-register dependencies?
2. When registering that dependency, why does the kafka-streams app end up with that namespaced type name?
3. Why does the schema registry consider those fields as having different types? Can it not understand the namespaced references?

Answer 1

The solution in my case was just to not pre-register the schemas, and instead start from a clean schema registry. The kafka-streams app auto-registered the relevant schemas.

I am guessing that the way I registered the original schema wasn't quite correct.