Search code examples
javaapache-kafkakerberosmessage-hubkcat

How to use kafkacat with message-hub

In trying to use kafkacat with message hub, I've used the following:

kafkacat -X client.id=xxxxx \
-X sasl.jaas.config='org.apache.kafka.common.security.plain.PlainLoginModule required username="xxxx" password="xxxx";' \
-X security.protocol=SASL_SSL \
-X sasl.mechanism=PLAIN \
-X ssl.protocol=TLSv1.2 \
-X ssl.enabled.protocols=TLSv1.2 \
-X ssl.truststore.type=JKS \
-X ssl.endpoint.identification.algorithm=HTTPS -b broker:port -C -t topic

and got the following error:

% ERROR: Java JAAS configuration is not supported, see https://github.com/edenhill/librdkafka/wiki/Using-SASL-with-librdkafka for more information.

How do I use kerberos with the provided credentials to create the correct keys, and then use those to consume from a topic?

Solution

  • You cannot use the Java argument names with kafkacat.

    You need to use the librdkafka configuration names instead. For example, the following will run a Consumer:

    kafkacat -X "security.protocol=sasl_ssl" -X 'sasl.mechanisms=PLAIN' \
  -X 'sasl.username=token' -X "sasl.password=<APIKEY>" \
  -X "ssl.ca.location=<CERTS_PATH>" -b <BOOTSTRAP_SERVERS> -C -t <TOPIC>

    The configuration specified in the example are good for Message Hub.

    <CERTS_PATH> depends on your operating system:

    • macOS: /etc/ssl/cert.pem
    • Debian: /Ubuntu/IBM Cloud: /etc/ssl/certs/
    • RedHat: /etc/pki/tls/cert.pem