Search code examples
prometheusgrafanagrafana-alerts

How to add a prometheus label to a field in alert rule

I have a Prometheus that collects statistics from envoy. I have several booths (for testing, development and production) all running envoy.

So I have quite a lot of labels for any query of the type:

envoy_cluster_upstream_rq_total - a query which will return the number of requests for all clusters on all stands.

So I have labels:

  1. envoy_cluster_name
  2. job
  3. instance

Now I'm setting up the alert rules and I have a little problem.

I need the alert to link to my dashboard. I am using template variables inside Grafana and so my link should look something like this:

https://example_stand/d/heHhNSFf6Na8vIZWRs8H/example_dashboard?orgId=1&refresh=10s &var-datasource=Prometheus&var-cluster={envoy_cluster_name}&var-job={job}&var-instance=All&var-min=1m

Where var-cluster and other var are my template variables. I want to somehow pass envoy_cluster_name and job to var-cluster and var-job respectively.

Is this somehow possible?

P.s I tried {$envoy_cluster_name} and {{envoy_cluster-name}} but this syntax didn't help me

Solution

  • All in all, it's very simple.

    The tags themselves are added as in the message template. You need to specify

    {{ .Labels.envoy_cluster_name }} and {{ .Labels.job }}

    So the final link will look something like this:

    https://example_stand/d/heHhNSFf6Na8vIZWRs8H/example_dashboard?orgId=1&refresh=10s &var-datasource=Prometheus&var-cluster={{ .Labels.envoy_cluster_name }}&var-job={{ .Labels.job }}&var-instance=All&var-min=1m