Search code examples
monitoringprometheusprometheus-alertmanager

Prometheus AlertManager - Send Alerts to different clients based on routes


I have 2 services A and B which I want to monitor. Also I have 2 different notification channels X and Y in the form of receivers in the AlertManager config file.

I want to send to notify X if service A goes down and want to notify Y if service B goes down. How can I achieve this my configuration?

My AlertManager YAML file is:

route:
  receiver: X

receivers:
  - name: X
    email_configs:

  - name: Y
    email_configs:

And alert.rule files is:

groups:

- name: A
  rules:
    - alert: A_down
      expr: expression
      for: 1m
      labels:
         severity: critical
      annotations:
         summary: "A is down"

- name: B
  rules:
    - alert: B_down
      expr: expression
      for: 1m
      labels:
        severity: warning
      annotations:
        summary: "B is down"

Solution

  • The config should roughly look like this (not tested):

        route:
          group_wait: 30s
          group_interval: 5m
          repeat_interval: 2h
    
          receiver: 'default-receiver'
    
          routes:
          - match:
              alertname: A_down
            receiver: X
          - match:
              alertname: B_down
            receiver: Y
    
    The idea is, that each [`route`](https://prometheus.io/docs/alerting/configuration/#%3Croute%3E) field can has a `routes` field, where you can put a different config, that gets enabled if the labels in `match` match the condition.