Search code examples
grafanagrafana-templatinggrafana-alerts

How do I get metric names in Grafana's message template?

I'm setting up Grafana v8.3.4 alerts. If I use a template like the following

{{ define "test_message" }}

{{ range .Alerts.Firing }}
{{ print .ValueString }} 
{{ end }}

{{ end }}

enter image description here

And test the e-mail notification using this template and the predefined message:

enter image description here

I get:

[ metric='foo' labels={instance=bar} value=10 ]

In the example above there's only one metric, but there could be more.

Question: Is there any way to loop through the returned metrics and display it as a table in the e-mail?

It would look something like:

Metric Name Value
foo 10
x 22
y 7

...

[UPDATE 6/24/2022] @Luis Almeida's answer led me to the right path, so I'm marking it as the correct answer even though my final solution was a little different.

Here are a couple of other links that also helped:

How to template annotations and labels - Grafana's Official Documentatio n Unified Alerting Grafana 8 | Prometheus | Victoria | Telegraf | Notifications | Alert Templating

Solution

  • So what you basically have are multiple alerts (which one has a metric defined by [ metric='foo' labels={instance=bar} value=10 ]) and you want to pull all this alerts in a table?

    I'm using a classic condition with a median function in order to remove eventual outliers btw. If you want to access the value in:

    • the first condition you should put {{ $values.B0.Value }}
    • the second condition you should put {{ $values.B1.Value }}
    • the n-th condition you should put {{ $values.Bn.Value }} classic condition If you want to access the a normal query value I think the variable {{ $values.A.Value }} where A is the query's name, but since grafana documentation for versions below 9 disappear I cannot confirm that

    You can use annotations to give a more readable metric's name as depicted in the image above.

    I believe that html tags are escaped since html tags appears on the template as text. So now that you have the annotations with the given values you have to edit the default email template that is in /usr/share/grafana/public/emails/ng_alert_notification.html

    Warning: Do a backup of this file just in case something goes wrong

    In the image bellow is how annotations are render in the email, now you just have to substitute that code with some code that renders a table. I believe that now it's fairly simple to do it yourself. (I didn't put the code here since I am not that comfortable with golang). If you want something different you can analise this file and change as you want. ng_alert_notification Note: To take effect the changes done in ng_alert_notification.html you have to restart grafana server

    Good looking HTML and CSS example

    Note: I'm using v8.5.2