How to fix the error undefined variable "$labels"in Prometheus?

Question

I used this alret

        - alert: my alert
          expr: status{status="ERROR"}
          for: 30m
          labels:
            severity: WARNING   
          annotations:
            myData: "{{ $labels.myData }}"
            myData2: "{{ $labels.myData2 }}"

I got an error ERROR - templates/: parse error in "prometheus/templates/alertmanager-prometheusRule.yaml": template: prometheus/templates/alertmanager-prometheusRule.yaml:419: undefined variable "$labels"

I saw the same issue in

Prometheus Docker fails to start with `Template: (dynamic): parse: template: :10: undefined variable "$labels"`

but I didn't understand how to solve it

in the configuration I used this data

text: "{{ range .Alerts -}}{{ .Annotations.myData }}{{ .Annotations.myData2}}{{ end-}}"

The error is from helm lint

Answer 1

It seems you are deploying your Prometheus setup via a helm chart. This causes an issue as the same delimiters ({{ and }}) are used both by helm templating and the alerting templating in Prometheus.

The {{ $labels.myData }} has to reach prometheus config intact, so helm must not process it.

The simplest way would be to use:

{{ "{{" }} $labels.myData }}

The {{ "{{" }} block will be processed by helm and will produce {{ as a result with rest of the line not being altered and will get you the result you need.