How to use different load balancer ports for Kafka bootstrap and brokers created by Strimzi?

Question

I have a local k3s Kubernetes cluster created by Rancher Desktop.

I am trying to set up Kafka using load balancer based on this tutorial to make it accessible for clients running outside of Kubernetes.

Here are my steps:

kubectl create namespace hm-kafka
kubectl apply --filename="https://strimzi.io/install/latest?namespace=hm-kafka" --namespace=hm-kafka
kubectl apply --filename=my-kafka-persistent.yaml --namespace=hm-kafka

my-kafka-persistent.yaml (based on kafka-persistent.yaml):

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: hm-kafka
spec:
  kafka:
    version: 3.4.0
    replicas: 3
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
      - name: external
        port: 9094
        type: loadbalancer
        tls: false
        configuration:
          brokers:
            - broker: 0
              advertisedPort: 9095
            - broker: 1
              advertisedPort: 9096
            - broker: 2
              advertisedPort: 9097
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      default.replication.factor: 3
      min.insync.replicas: 2
      inter.broker.protocol.version: "3.4"
    storage:
      type: jbod
      volumes:
        - id: 0
          type: persistent-claim
          size: 100Gi
          deleteClaim: false
  zookeeper:
    replicas: 3
    storage:
      type: persistent-claim
      size: 100Gi
      deleteClaim: false
  entityOperator:
    topicOperator: {}
    userOperator: {}

When I deploy, I got this issue

Pods:

Services:

This is because advertisedPort option in my YAML file doesn’t really change the port used in the load balancer itself. It changes only the port number used in the advertised.listeners Kafka broker configuration parameter.

And they all try to use the port 9094.

I am wondering is there a way I can change load balancer port so that bootstrap and brokers have different ports? Will it help make Kafka work? Thanks!

Also, I read another similar question and its answer, does it mean which port to use depends on k3s Kubernetes cluster created by Rancher Desktop and we have no control? Thanks!

Answer 1

To allow clients to access the individual brokers, Strimzi needs to create one type: LoadBalancer service for every broker. In addition, it creates one more service targeting all brokers for bootstrapping (that can be disabled). This normally results in Kubernetes provisioning one load balancer per service -> so there is no need to do any complicated hacks with port numbers.

If your environment does not support load balancers like this, maybe the best thing is to try one of the other mechanisms for exposing the Kafka cluster. For example NodePorts or Kubernetes Nginx Ingress.

You can also do this your self if you want:

• Configure a regular internal listener instead of the type: loadbalancer one
• Use the advertisedHost and advertisedPort options to configure the hostname and port you want to use for each broker
• Create your own services with any configuration (e.g. port) you want (make sure the services have some custom name and do not have the Strimzi labels to avoid having them deleted or overwritten by Strimzi)