Spring boot kafka duplicated properties in producer, consumer, admin

Question

Spring boot Kafka provides in application.yml for some properties several placements for the configuration. For example ssl configuration:

# application.yml
spring:
  kafka:
    ssl:
      trust-store-location: 
       ...
    admin:
      ssl:
        trust-store-location: 
         ...
    producer:
      ssl:
        trust-store-location: 
        ...
    consumer:
      ssl:
        trust-store-location: 
        ...

All of them use the same class KafkaProperties to configure. But might KafkaProperties be different for producer/consumer/admin beans here? Am I right that if producer/consumer/admin miss their own properties then they use ones from the base place spring.kafka.ssl? And if producer/consumer/admin have their own properties they will ignore ones from the base spring.kafka.ssl?

Answer 1

The ssl.truststore.location default value is null. As such, i am using another property, min.insync.replicas as example, but the concept is the same.

But might KafkaProperties be different for producer/consumer/admin beans here?

You can configure properties at a granular level if required. For example, for a KafkaAdmin, this is how you do it:

@Bean
public KafkaAdmin kafkaAdmin() {
    Map<String, Object> configs = new HashMap<>();

    //other configs here
    configs.put("spring.kafka.admin.properties.min.insync.replicas", 1);
    
    return new KafkaAdmin(configs);
}

Am I right that if producer/consumer/admin miss their own properties then they use ones from the base place spring.kafka.ssl?

If the bean specific properties is not configured(as per my example above), Spring Kafka will look in the application.yaml(Common properties can be found here), for example:

spring
  kafka
    admin
      properties
        min-insync-replicas: 3

And if it is not there, the default will be configured. One way to find the default is by issuing this command (min.insync.replicas is at broker/topic level):

kafka-configs.bat --bootstrap-server localhost:9092 --entity-type topics --entity-name <topic-name> --describe --all

//Result:
min.insync.replicas=1 sensitive=false synonyms={DEFAULT_CONFIG:min.insync.replicas=1}

The github's page of Spring Kafka TopicBuilder also specifies this:

Since 2.6 partitions and replicas default to * {@link Optional#empty()} indicating the broker defaults will be applied.

which is 1 as per Kafka's broker default, as per here.

And if producer/consumer/admin have their own properties they will ignore ones from the base spring.kafka.ssl?

Right.