Enable Authentication on strimzi kafka

Question

i'm tyring to enable authentication on strimzi kafka. below is my yml

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    version: 3.2.1
    replicas: 1
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
        authentication:
          type: scram-sha-512
          usernameSecretRef:
            name: kafka-user-auth
            key: kafka-user
          passwordSecretRef:
            name: kafka-user-auth
            key: kafka-pass
    resources:
      requests:
        memory: 1Gi
        cpu: 0.4
      limits:
        memory: 1Gi
        cpu: 0.5
    template:
      pod:
        tolerations:
          - key: kubernetes.azure.com/scalesetpriority
            operator: Equal
            value: spot
            effect: NoSchedule
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
      default.replication.factor: 1
      min.insync.replicas: 1
      inter.broker.protocol.version: '3.1'
    storage:
      type: persistent-claim
      size: 10Gi
      deleteClaim: true
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-metrics
          key: kafka-metrics-config.yml
  zookeeper:
    replicas: 1
    storage:
      type: persistent-claim
      size: 2Gi
      deleteClaim: true
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-metrics
          key: zookeeper-metrics-config.yml
  kafkaExporter:
    topicRegex: .*
    groupRegex: .*

but when applying this yml, i get below error error: error validating "strimzi-kafka-with-auth.yml": error validating data: [ValidationError(Kafka.spec.kafka.listeners[1].authentication): unknown field "passwordSecretRef" in io.strimzi.kafka.v1beta2.Kafka.spec.kafka.listeners.authentication, ValidationError(Kafka.spec.kafka.listeners[1].authentication): unknown field "usernameSecretRef" in io.strimzi.kafka.v1beta2.Kafka.spec.kafka.listeners.authentication]; if you choose to ignore these errors, turn validation off with --validate=false

i tried different approaches, e.g.

authentication:
      type: scram-sha-512
      username: myuser
      passwordSecret:
        secretName: myuser-secret
        passwordKey: password

and this

authentication:
  type: sasl_plaintext
  usernameSecret:
    secretKeyRef:
      name: myuser-secret
      key: myuser-username
  passwordSecret:
    secretKeyRef:
      name: myuser-secret
      key: myuser-password

and this

authentication:
          type: scram-sha-512
          username: my-kafka-user
          password: my-kafka-password

but none of the above approaches seem to work, i get errors like it's not recognizing either username or password fields or not recognizing usernameSecret and passwordSecret fields

UPDATE

i am now creating kafka using below yml which works fine

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    version: 3.2.1
    replicas: 1
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
        authentication:
          type: scram-sha-512
    resources:
      requests:
        memory: 1Gi
        cpu: 0.4
      limits:
        memory: 1Gi
        cpu: 0.5
    template:
      pod:
        tolerations:
          - key: "kubernetes.azure.com/scalesetpriority"
            operator: "Equal"
            value: "spot"
            effect: "NoSchedule"
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
      default.replication.factor: 1
      min.insync.replicas: 1
      inter.broker.protocol.version: "3.1"
    storage:
      type: ephemeral
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-metrics
          key: kafka-metrics-config.yml
  zookeeper:
    replicas: 1
    storage:
      type: ephemeral
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-metrics
          key: zookeeper-metrics-config.yml
  kafkaExporter:
    topicRegex: ".*"
    groupRegex: ".*"
  entityOperator:
    topicOperator: {}
    userOperator: {}

and a user using user operator

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  name: my-user
  labels:
    strimzi.io/cluster: my-cluster
spec:
  authentication:
    type: scram-sha-512

now when i try to connect kafka client, it requires ssl certificates. however i want kafka broker to authenticate the client using username/pass and transport data over tls without certificate verification. is it possible? i don't want any certificates to be placed on client side

Answer 1

The Kafka custom resource does not configure any passwords or users. It only configures the authentication type. The actual users and their credentials are separate from that. You can create users for example using the KafkaUser resources through the User operator. Or you can do it directly in Kafka using Kafka Admin APIs if you don't want to use the User Operator. Or with some authentication methods such as OAuth, the users live somewhere else completel.

You can check out the examples: https://github.com/strimzi/strimzi-kafka-operator/tree/main/examples/security and / or read the docs which have some examples as well (e.g. https://strimzi.io/docs/operators/latest/full/deploying.html#assembly-securing-kafka-str). You can also check the API reference which shows which fields actually exist and can be used.