Search code examples
kubernetesapache-kafkaapache-kafka-connectconfluent-platform

How can i set PDB (Pod Disruption Budget) for CFK Kafka connect resource


I am using confluent cloud and then have Connect cluster using CFK

https://github.com/confluentinc/confluent-kubernetes-examples/tree/master/hybrid/ccloud-integration

Its all working fine but i am not sure how can i set PDB as i have 8 replicas and i use karpenetr to scale nodes. I have to setup PDB to make sure atleast 7 replicas run at all times

---
apiVersion: platform.confluent.io/v1beta1
kind: Connect
metadata:
  name: connect
  namespace: confluent
spec:
  replicas: 1
  image:
    application: confluentinc/cp-server-connect:7.4.0
    init: confluentinc/confluent-init-container:2.6.0
  tls:
    autoGeneratedCerts: true
  mountedSecrets:
    - secretRef: kafka-client-config-secure
  dependencies:
    kafka:
      bootstrapEndpoint: <cloudKafka_url>:9092
      authentication:
        type: plain
        jaasConfig:
          secretRef: cloud-plain
      tls:
        enabled: true
        ignoreTrustStoreConfig: true 
    schemaRegistry:
      url: https://<cloudSR_url>
      authentication:
        type: basic
        basic:
          secretRef: cloud-sr-access
---

Solution

  • you can use something like this;

    ---
    apiVersion: platform.confluent.io/v1beta1
    kind: Connect
    metadata:
      labels:
        app: connect
      name: connect
      namespace: confluent
    spec:
      replicas: 7
      image:
        application: confluentinc/cp-server-connect:7.4.0
        init: confluentinc/confluent-init-container:2.6.0
      tls:
        autoGeneratedCerts: true
      mountedSecrets:
        - secretRef: kafka-client-config-secure
      dependencies:
        kafka:
          bootstrapEndpoint: <cloudKafka_url>:9092
          authentication:
            type: plain
            jaasConfig:
              secretRef: cloud-plain
          tls:
            enabled: true
            ignoreTrustStoreConfig: true 
        schemaRegistry:
          url: https://<cloudSR_url>
          authentication:
            type: basic
            basic:
              secretRef: cloud-sr-access
    ---
    
    apiVersion: policy/v1
    kind: PodDisruptionBudget
    metadata:
      name: connect-pdb
      namespace: confluent
    spec:
      minAvailable: 7
      selector:
        matchLabels:
          app: connect