Search code examples
javaapache-kafkaconfluent-schema-registry

auto.register.schemas set to false doesn't work as intended


auto.register.schemas=false

doesn't work as I expect.

If I read the documentation it's suppose to counter the producer to regsiter new schemas.

https://docs.confluent.io/platform/current/schema-registry/schema_registry_onprem_tutorial.html#auto-schema-registration

https://docs.confluent.io/platform/current/schema-registry/serdes-develop/index.html#handling-differences-between-preregistered-and-client-derived-schemas

schema-registry.properties

#     listeners = PLAINTEXT://your.host.name:9092
listeners=http://0.0.0.0:8081

# Use this setting to specify the bootstrap servers for your Kafka cluster and it
# will be used both for selecting the leader schema registry instance and for storing the data for
# registered schemas.
kafkastore.bootstrap.servers=192.168.16.192:9092,192.168.16.191:9092

# The name of the topic to store schemas in
kafkastore.topic=_schemas

# If true, API requests that fail will include extra debugging information, including stack traces
debug=false

auto.register.schemas=false
use.latest.version=true

To be sure some previous set up aren't persisted I deleted the _schemas topic for every attempt and changed the name as well.

But still, every time I produce some kafka message I see that a new schmas named < topic-name >-value is registered.

I don't understand why.

The only way it worked is when I added:

props.put(AbstractKafkaAvroSerDeConfig.AUTO_REGISTER_SCHEMAS, false);

Then I had the error Error retrieving Avro schema.

But I should have this error also with the auto.register.schemas property set to false right ?

My producer code is:

Properties props = new Properties();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "<broker-address>");
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, KafkaAvroSerializer.class);
props.put(ProducerConfig.CLIENT_ID_CONFIG, "Kafka Avro  Producer");
props.put("schema.registry.url", "<schema-registry>");
#props.put(AbstractKafkaAvroSerDeConfig.AUTO_REGISTER_SCHEMAS, false);

KafkaProducer<String, ClientOrderRequest> producerRequest = new KafkaProducer<>(props);

ClientOrderRequest clientOrderRequest = createClientOrderRequest();
            
final ProducerRecord<String, ClientOrderRequest> producerOrderRequest = new ProducerRecord<>("client-order-request",
                    "ClientOrderRequest-" + calendar.getTimeInMillis(), clientOrderRequest);
producerRequest.send(producerOrderRequest);

Solution

  • My mistake is that :

    auto.register.schemas=false
    use.latest.version=true
    

    Are kafka client properties and not schema-registry properties. Those need to be set on the Producer side.