Kafka to zookeeper DIGEST-MD5 authentication error

Question

I could able to do DIGEST-MD5 authentication with kafka broker and embedded zookeeper which is given by Kafka but I was getting following error message while doing DIGEST-MD5 authentication with Kafka broker and non embedded zookeeper (separate zookeeper instance).

Please find below is the error message:

[2018-11-05 19:44:21,536] ERROR SASL authentication failed using login context 'Client' with exception: {} (org.apache.zookeeper.client.ZooKeeperSaslClient)

javax.security.sasl.SaslException: Error in authenticating with a Zookeeper Quorum member: the quorum member's saslToken is null.

org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = AuthFailed for /consumers

Answer 1

What your jaas files look like ?

For Zookeeper cluster, you must have three sections in zookeeper jaas file : Server, QuorumServer and QuorumLearner

QuorumServer and QuorumLearner sections are used to authenticate zookeeper servers to each other. See example of configuration here in §DIGEST-MD5 based authentication

Server section is used to authenticate kafka brokers. It should contains kafka username and password formatted as shown here in §Server Configuration/ JAAS configuration file: DIGEST-MD5 authentication

Your Kafka jaas file should have a Client section with a username and a password as shown here in §Client Configuration / JAAS configuration file: DIGEST-MD5 authentication

Finally, check that you are using correct environment variable for zookeeper jaas file :

SERVER_JVMFLAGS="-Djava.security.auth.login.config=/path/to/server/jaas/file.conf"

Embedded zookeeper uses KAFKA_OPTS