Spring Boot + Kafka + Kerberos configuration

Question

I am using Spring Boot 1.5.6.RELEASE to connect to Kafka 0.11 using Kerberos authentication. These are the dependencies I'm using for Kafka:

        <dependency>
            <groupId>org.springframework.integration</groupId>
            <artifactId>spring-integration-kafka</artifactId>
            <version>3.0.0.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
            <version>2.0.8.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-stream-kafka</artifactId>
            <version>2.1.1.RELEASE</version>
        </dependency>

I have to send messages to a Kafka server that is not under our management and I was given a Kafka username, keytab file and a krb5.conf file.

These are the properties used for testing without Kerberos:

spring:
  kafka:
    bootstrap-servers: "10.10.20.185:9092"
    producer:
      value-serializer: org.springframework.kafka.support.serializer.JsonSerializer

Works fine.

How do I implement Kerberos in my app configuration? As I am new to Kafka and Kerberos, any help would be appreciated.

Answer 1

See the kafka documentation "Authentication using SASL/Kerberos".

To configure SASL authentication on the clients: Clients (producers, consumers, connect workers, etc) will authenticate to the cluster with their own principal (usually with the same name as the user running the client), so obtain or create these principals as needed. Then configure the JAAS configuration property for each client. Different clients within a JVM may run as different users by specifiying different principals. The property sasl.jaas.config in producer.properties or consumer.properties describes how clients like producer and consumer can connect to the Kafka Broker. The following is an example configuration for a client using a keytab (recommended for long-running processes):

sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \
    useKeyTab=true \
    storeKey=true  \
    keyTab="/etc/security/keytabs/kafka_client.keytab" \
    principal="[email protected]";

For command-line utilities like kafka-console-consumer or kafka-console-producer, kinit can be used along with "useTicketCache=true" as in:

sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \
    useTicketCache=true;

JAAS configuration for clients may alternatively be specified as a JVM parameter similar to brokers as described here. Clients use the login section named KafkaClient. This option allows only one user for all client connections from a JVM.

Make sure the keytabs configured in the JAAS configuration are readable by the operating system user who is starting kafka client. Optionally pass the krb5 file locations as JVM parameters to each client JVM (see here for more details):

-Djava.security.krb5.conf=/etc/kafka/krb5.conf

Configure the following properties in producer.properties or consumer.properties:

security.protocol=SASL_PLAINTEXT (or SASL_SSL)
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka