Search code examples
dockerapache-kafkadocker-composejaassasl

docker-compose wurstmeister/kafka failing to parse KAFKA_OPTS

I have a basic docker-compose file file for wurstmeister/kafka

I'm trying to configure it to use SASL_PLAIN with SSL However I keep getting this error no matter how many ways I try to specify my jaas file

This is the error I get

[2018-04-11 10:34:34,545] FATAL [KafkaServer id=1001] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.IllegalArgumentException: Could not find a 'KafkaServer' or 'sasl_ssl.KafkaServer' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set

These are the vars I have. Last one is where I specify my jaas file

environment:
  KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
  KAFKA_HOST_NAME: 10.10.10.1
  KAFKA_PORT: 9092
  KAFKA_ADVERTISED_PORT: 9093
  KAFKA_ADVERTISED_HOST_NAME: 10.10.10.1
  KAFKA_LISTENERS: PLAINTEXT://:9092,SASL_SSL://:9093
  KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://10.10.10.1:9092,SASL_SSL://10.10.10.1:9093
  KAFKA_SECURITY_INTER_BROKER_PROTOCOL: SASL_SSL
  KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
  SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
  KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
  KAFKA_SSL_TRUSTSTORE_LOCATION: /kafka.server.truststore.jks
  KAFKA_SSL_TRUSTSTORE_PASSWORD: password
  KAFKA_SSL_KEYSTORE_LOCATION: /kafka.server.keystore.jks
  KAFKA_SSL_KEYSTORE_PASSWORD: password
  KAFKA_SSL_KEY_PASSWORD: password
  KAFKA_OPTS: '-Djava.security.auth.login.config=/path/kafka_server_jaas.conf'

Also when I try to check the docker logs I see

/usr/bin/start-kafka.sh: line 96: KAFKA_OPTS=-Djava.security.auth.login.config: bad substitution

Any help is greatly appreciated!

Solution

  • Update: They have fixed it and it is merged now!

    https://github.com/wurstmeister/kafka-docker/pull/321

    There's a bug open now with wurstmeister/kafka but they have gotten back to me with a workaround as follows

    I believe his is part of a larger namespace collision problem that affects multiple elements such as Kubernetes deployments etc (as well as other KAFKA_ service settings).

    Given you are referencing an external file /kafka_server_jaas.conf, i'm assuming you're OK adding/mounting extra files through; a work-around is to specify a CUSTOM_INIT_SCRIPT environment var, which should be a script similar to:

    #!/bin/bash
export KAFKA_OPTS="-Djava.security.auth.login.config=/kafka_server_jaas.conf"
This is executed after the substitution part that is failing.

    This could have been done inline, however there is currently a bug in how we process the environment, where we need to specify the input separator to make this work correctly.

    Hopefully this works!