Search code examples
spring-bootapache-kafkaspring-kafkakafka-producer-api

Kafka Producer SSL properties for YAML file


This is the Kafka Producer properties in YAML file. When i am enabling SSL my kafka producer doesn't work.Its not able to identify the topic on the broker .But when i use PLAINTEXT my kafka producer works properly. Am i missing something for SSL config.

PS: Bootsrap server are different for SSL and PLAINTEXT.

spring:
   kafka:
      producer:
        bootstrap-servers: <server name>
        properties:
          acks: all
          retries: 3 
          retry.backoff.ms: 200000
      ssl.protocol: SSL
      ssl.endpoint.identification.algorithm: https 
      ssl:
        keystore-location: keystore.jks
        keystore-password: password

This is my Kafka Producer config

@Bean
    public ProducerFactory<String, JsonMessage> producerFactory() {
        Map<String, Object> config = new HashMap<>();

        config.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
        config.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        config.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);
        config.put(ProducerConfig.ACKS_CONFIG, acks);
        config.put(ProducerConfig.RETRIES_CONFIG, retries);
        config.put(ProducerConfig.RETRY_BACKOFF_MS_CONFIG, retryBackoffMs);
        
        


        return new DefaultKafkaProducerFactory<>(config);
    }

    @Bean
    public KafkaTemplate<String, JsonMessage> kafkaTemplate() {
        return new KafkaTemplate<>(producerFactory());
    }

This is the values returned for kafka prodcuer on spring boot console

ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
 ssl.secure.random.implementation = null

Solution

  • You are creating your own ProducerFactory bean so the properties in application.yml are not being used; those properties are used by Boot when auto configuring its bean.

    You need to set the SSL properties yourself in your producerFactory() bean.