Strimzi kafka - Topic authorization exception on non-tls

Question

i have a Strimzi kafka cluster, deployed using the following yaml. LoadBalancer is enabled on port 9094, without TLS

I've a KafkaTopic created, and when i try to produce to the Topic on port 9094, it gives TopicAuthorization exception.

# KafkaDeployment.yaml
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: dataproc-poc #1
spec:
  kafka:
    version: 3.0.0
    replicas: 3
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
      - name: external
        port: 9094
        type: loadbalancer
        tls: false 
    authorization:
      type: simple
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      log.message.format.version: "3.0"
      inter.broker.protocol.version: "3.0"
    storage:
      type: jbod
      volumes:
      - id: 0
        type: persistent-claim
        size: 2Gi
        deleteClaim: false
    logging: #9
      type: inline
      loggers:
        kafka.root.logger.level: "INFO"
  zookeeper:
    replicas: 3
    storage:
      type: persistent-claim
      size: 2Gi
      deleteClaim: false
    resources:
      requests:
        memory: 1Gi
        cpu: "1"
      limits:
        memory: 2Gi
        cpu: "1.5"
    logging:
      type: inline
      loggers:
        zookeeper.root.logger: "INFO"
  entityOperator: #11
    topicOperator: {}
    userOperator: {}


# kafka-topic.yaml
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
  name: my-topic
  labels:
    strimzi.io/cluster: dataproc-poc
spec:
  partitions: 3
  replicas: 3
  config:
    retention.ms: 7200000
    segment.bytes: 1073741824


apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  name: my-topic
  labels:
    strimzi.io/cluster: dataproc-poc
spec:
  authentication:
    type: scram-sha-512
  authorization:
    type: simple
    acls:
    # Topics and groups used by the HTTP clients through the HTTP Bridge
    # Change to match the topics used by your HTTP clients
    - resource:
        type: group
        name: mygroup
      operation: Read
    - resource:
        type: topic
        name: my-topic
        patternType: literal
      operation: Write
          
---
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaBridge
metadata:
  name: my-topic
spec:
  replicas: 1
  bootstrapServers: dataproc-poc-kafka-bootstrap:9092
  http:
    port: 8080

When i try to access the Topic using commandline, it gives the error as shown below:

Karans-MacBook-Pro:dataproc-poc karanalang$ $CONFLUENT_HOME/bin/kafka-console-producer --broker-list 34.75.244.133:9094 --topic my-topic
>hi therr
[2022-01-30 21:59:47,985] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 3 : {my-topic=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)
[2022-01-30 21:59:48,008] ERROR [Producer clientId=console-producer] Topic authorization failed for topics [my-topic] (org.apache.kafka.clients.Metadata)
[2022-01-30 21:59:48,012] ERROR Error when sending message to topic my-topic with key: null, value: 8 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [my-topic]

Any ideas what i need to do to fix this ? tia !

Answer 1

None of your listeners has enabled authentication. So when you connect to it, you are just authenticated as ANONYMOUS. And ANONYMOUS has no ACLs, so it is not allowed to do anything. So you either need to enable authentication and use it or you need to disable the authorization. You should also always check the broker logs where you get the full authroization error with all details and make it more clear what the issue is.