Unexpected Kafka request of type METADATA during SASL handshake

Question

I am trying to connect Kafka Java Client to a Kafka broker using SASL Plain. But when I try to send a message from the Producer, the Kafka Server logs the following error:

[2020-04-30 14:48:14,955] INFO [SocketServer brokerId=0] Failed authentication with /127.0.0.1 (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)

By the looks, the producer tries to send a metadata request, before the SASL handshake. How can I do the handshake before sending the message?

Following is my kafka_server_jaas.conf file, which is used for Kafka Server.

KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin-secret"
    user_admin="admin-secret";
};

Client {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin-secret";
};

Following is my zookeeper_jaas.conf file, which is used for the zookeeper:

Server {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin-secret"
    user_admin="admin-secret";
};

In my Java producer, I set the following properties:

Properties properties = new Properties();
properties.put("bootstrap.servers", "localhost:9092");
properties.put("sasl.jaas.config", "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"admin_secret\"");
properties.put("sasl.mechanisms", "PLAIN");
properties.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
KafkaProducer kafkaProducer = new KafkaProducer(properties);

Is there anything wrong I am doing?

Answer 1

You need to specify security.protocol, otherwise by default, Kafka clients do not use SASL.

In your client properties, add:

properties.put("security.protocol", "SASL_SSL");

There's also SASL_PLAINTEXT but it's not recommended to use the PLAIN mechanism over SASL_PLAINTEXT as effectively the password will be exchanged in cleartext.