Dynamic data in values.yaml file

I am trying to find a way to populate values.yaml with values I get from a DB tables.

I have values.yaml with values like this:

...
my_image:
  repository: my_repo
  replicaCount: 1
  tag: latest
  name: my_name
...

In my template.yaml file I have this:

...
  - image: "{{ .Values.my_image.repository }}:{{ .Values.my_image.tag }}"
...

I can create a json or yaml file from the query result. I know that I can not populate the values.yaml without using tpl. But I am not sure how to? Is my only option to recreate the template.yaml every time?

Is there a way to generate tagvalues.yaml with a map of tags? And use it like this in values.yaml file:

...
my_image:
  repository: my_repo
  replicaCount: 1
  tag: tagvalues.tag
  name: my_name
...

Solution

You can use the helm install -f file to supply an additional Helm values file when you go to install a chart. This can be any YAML (or JSON) file, and the values here override what's in the chart's values.yaml.

So in your chart, set values.yaml to have reasonable defaults, or use conditionals or the default template function for optional values, or use the required template function if there is no reasonable default.

# values.yaml
my_image:
  # repository: must be provided at install time (for the sake of example)
  replicaCount: 1
  # tag: latest (optional)

# templates/deployment.yaml
spec:
  replicas: {{ .Values.my_image.replicaCount }}
  template:
    spec:
      containers:
{{- $repository := .Values.my_image.repository | required "my_image: { repository: } is required"}}
{{- $tag := .Values.my_image.tag | default "latest" }}
        - image: "{{ $repository }}:{{ $tag }}"

Write a script in your favorite language that connects to the database and produces values-file output. It doesn't need to include all of the settings, only the ones you'll need to change from the chart's defaults. (I've specifically done things like this in Groovy in a Jenkins pipeline, Python or Go or anything else where you can connect to a database and write JSON will also work.)

$ ./find-latest-deployment.py --config dbconfig.toml -o deploy.json
$ cat deploy.json
{"my_image":{"repository":"replica.example.com","tag":"20241008"}}

And now that you've got that, you can pass it as Helm values.

helm upgrade --install myapp ./chart -f deploy.json

Actually writing the script is left as an exercise for the reader. Note that Helm can't directly run it, either as an input or from the template code, so this needs to leave completely separately from the Helm logic; but on the flip side that also means you can deal with things like database credentials in whatever way is convenient for your environment. You'll also need to set up your deployment pipeline to first run this script and then run Helm, and again that depends on specifics of the pipeline.