Search code examples
templatesapache-kafkayamlrangekubernetes-helm

Helm range yaml template kafka topics


I am new to helm and I am trying to generate different topics for kafka with a range function to not have a yaml file for each topic:

I have different topics (topic1, topic2, topic3,...) and the only difference they have is the retention in ms of the topic and the name, some topics have 3600000 and the others 540000, this is my values file:

KafkaTopics:
  shortRetentionTopics:
    name:
    - topic1
    - topic2
    - topic3
    - topic4
    spec:
      config: 
        retention.ms: 540000
      topicName:
      - topic1logs
      - topic2logs
      - topic3logs
      - topic4logs
  longRetentionTopics:
    name:
    - topic34
    - topic35
    spec:
      config: 
        retention.ms: 3600000
      topicName:
      - topic34logs
      - topic34logs

And I would like to set the name, topicName and retention.ms on this template doing a for loop from the values file:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
  labels:
    strimzi.io/cluster: kafka
  name: (here the name of the topic)
  namespace: default
spec:
  config:
    retention.ms: (here the retention of the topic)
  partitions: 12
  replicas: 1
  topicName: (here the topicName of the topic)

Or if you have any suggestion to change the structure of the values file to make it easier to parse the values to the template I'm interested as well.


Solution

  • At the end I ended up doing this:

    {{- range $topics := .Values.kafkaTopicList }}
    {{ $spec := default dict $topics.spec }}
    {{ $config := default dict $spec.config }}
    {{ $retention := default dict $config.retentionMs }}
    ---
    apiVersion: kafka.strimzi.io/v1beta2
    kind: KafkaTopic
    metadata:
      labels:
        strimzi.io/cluster: kafka
      name: {{ $topics.name }}
      namespace: default
    spec:
      config:
        retention.ms: {{ $retention | default "540000" }}
      partitions: 12
      replicas: 1
      topicName: {{ $topics.name | replace "-" "." }}
    {{- end}}
    

    Values file:

    kafkaTopicList:
      topic1:
        name: events-1
      topic2:
        name: events-2
      topic3:
        name: events-3
      topic4:
        name: events-4
      topic5:
        name: events-5
      topic6:
        name: events-6
      topic7:
        name: events-7
        spec:
          config: 
            retentionMs: 3600000