Spring cloud stream producer retries and error handling

Question

I have set a spring cloud stream kafka producer and consumer and there are 3 kafka brokers running . I have set min.insync.replicas to 4 to see how producer error handling works . The messagechannel.send call returns immediately and the producer logs keep saying NOT_ENOUGH_REPLICAS which is fine and expected .

server.port: 9050
spring:
  cloud:
    stream:
      bindings:
        errorChannel:
          destination: error-topic
        output:
          destination: stream-topic
          group: top-group
          producer:
            errorChannelEnabled: true
      kafka:
        bindings:
          output:
            producer:
              retries: 3
              sync: false
        binder:
          autoCreateTopics: true
          configuration:
            value:
              serializer: com.example.kafkapublisher.MySerializer
          producer-properties:
            acks: all
spring.cloud.stream.kafka.bindings.errorChannel.consumer.enableDlq: true
  

The above is my producer configuration . Although retries is set to 3 , the producer keeps on retrying a large number of times . Although sync is set to true , the send call comes out immediately . Although error channel and destination is defined , and errorChannelEnabled is set to true i do't see the failed message in the error topic my-error, neither is the error topic created . Request your help

Answer 1

Arbitrary Kafka producer properties go in the ...producer.configuration property.

https://docs.spring.io/spring-cloud-stream/docs/current/reference/html/spring-cloud-stream-binder-kafka.html#kafka-producer-properties

configuration

Map with a key/value pair containing generic Kafka producer properties. The bootstrap.servers property cannot be set here; use multi-binder support if you need to connect to multiple clusters.

Default: Empty map.