Configuration of 'sasl.jaas.config' when you have 2 separate topics, each having separate connection key?

Question

I am looking for instructions on the configuration of sasl.jaas.config when you have 2 separate topics, each having separate connection key? I am using spring-cloud-starter-stream-kafka version 3.1. I am not using spring-cloud-stream-binder-kafka and I don't know if that would solve my problem.

I can find lots of examples when there is only 1 topic involved, using a "general configuration in spring.cloud.stream.kafka.binder.configuration that is inherited by the consumer to connect.

I have a scenario where I have a input and output which are each on their own separate topic and I want to configure it in the .yml. Is this possible via .yml configuration?

My best guess was to try to configure the key spring.cloud.stream.bindings.input.configuration.sasl.jaas.config but that config doesn't seem to exist, does it?

Here is my best guess but it is not correct, since it doesn't work:

spring:
  ...
  cloud:
    stream:
      kafka:
        binder:
          brokers: ...
          defaultBrokerPort: 9093
          auto-create-topics: true
          configuration:
            security.protocol: SASL_SSL
            sasl:
              mechanism: PLAIN
        bindings:
          eeoi-sink:
            consumer:
              enableDlq: false
              dlqName: input_dlq
              ackEachRecord: true
              autoCommitOffset: false
      bindings:
        eeoi-sink:
          destination: input
          contentType: application/json
          group: $Default
          consumer:
            max-attempts: 1
          configuration:
            sasl:
              jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="...";
        acknowledgement-source:
          destination: output
          contentType: application/json
          group: $Default
          configuration:
            sasl:
              jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password="...";

Answer 1

spring.cloud.stream.bindings.input.configuration.sasl.jaas.config will not work. The correct property key is spring.cloud.stream.kafka.bindings.input.consumer.configuration.sasl.jaas.config.

If you want to use separate jaas config, you need to use a multi-binder setup. See this application for some ideas: https://github.com/spring-cloud/spring-cloud-stream-samples/blob/master/multi-binder-samples/kafka-multi-binder-jaas/src/main/resources/application.yml

In this sample, where it provides jaas config for the binder, you can simply replace it with binder.configuration.sasl.jaas.config.

Have the same Kafka cluster used as the broker for both binder setups, but different jaas configs.

However, I think the problem that you are running into is different. Your jaas config is ultimately delegated to java security Configuration which keeps a static copy of the jaas configuration per JVM. Thus, any jaas configs provided after it is set statically in the JVM are ignored.

See this for more details on a related issue: https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/874

This issue provides some workarounds for this problem. One is to make the connection string, password, etc. the same for both the topics, that way, the same jaas config values are used. This may not be an ideal solution and defeats the purpose of different credentials in the first place. The reason why the aforementioned sample works is because we use the same username/password combination in both binders.

Another option is to split the application into two. In this way, the jaas configuration for each topic resides in separate applications.