I am new to helm and I am trying to generate different topics for kafka with a range function to not have a yaml file for each topic:
I have different topics (topic1, topic2, topic3,...) and the only difference they have is the retention in ms of the topic and the name, some topics have 3600000 and the others 540000, this is my values file:
KafkaTopics:
shortRetentionTopics:
name:
- topic1
- topic2
- topic3
- topic4
spec:
config:
retention.ms: 540000
topicName:
- topic1logs
- topic2logs
- topic3logs
- topic4logs
longRetentionTopics:
name:
- topic34
- topic35
spec:
config:
retention.ms: 3600000
topicName:
- topic34logs
- topic34logs
And I would like to set the name, topicName and retention.ms on this template doing a for loop from the values file:
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
labels:
strimzi.io/cluster: kafka
name: (here the name of the topic)
namespace: default
spec:
config:
retention.ms: (here the retention of the topic)
partitions: 12
replicas: 1
topicName: (here the topicName of the topic)
Or if you have any suggestion to change the structure of the values file to make it easier to parse the values to the template I'm interested as well.
At the end I ended up doing this:
{{- range $topics := .Values.kafkaTopicList }}
{{ $spec := default dict $topics.spec }}
{{ $config := default dict $spec.config }}
{{ $retention := default dict $config.retentionMs }}
---
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
labels:
strimzi.io/cluster: kafka
name: {{ $topics.name }}
namespace: default
spec:
config:
retention.ms: {{ $retention | default "540000" }}
partitions: 12
replicas: 1
topicName: {{ $topics.name | replace "-" "." }}
{{- end}}
Values file:
kafkaTopicList:
topic1:
name: events-1
topic2:
name: events-2
topic3:
name: events-3
topic4:
name: events-4
topic5:
name: events-5
topic6:
name: events-6
topic7:
name: events-7
spec:
config:
retentionMs: 3600000