Search code examples
regexprometheusmonitoringprometheus-alertmanager

set alertmanager to distribute alerts to different channel by job name

I want to send my alert to two different distribution lists in Alertmanager for Prometheus. The only way to distinguish my alerts is by their job name.

my alert names are like below:

sample1:

Labels
alertname = SyslogErrors
instance = 22.32.23.32:2324
job = my-job-sample-service-dev
message = Exception raised during message subscription. Trying again in 60 seconds
monitor = server1
severity = critical
Annotations
description = Errors have been found for my-job-sample-service-dev application in /data/logs/messages/my-job-sample-service-dev syslog file
Source

sample2:

Labels
alertname = SyslogErrors
instance = 22.32.23.32:2324
job = my-job-sample-service-pre-dev
message = Exception raised during message subscription. Trying again in 60 seconds
monitor = server1
severity = critical
Annotations
description = Errors have been found for my-job-sample-service-pre-dev application in /data/logs/messages/my-job-sample-service-pre-dev syslog file
Source

here is my sample alertmanager config file:

global:
  smtp_smarthost: 'mail.server.com:25'
  smtp_from: '[email protected]'
  smtp_require_tls: false
templates:
- '/etc/alertmanager/template/*.tmpl'
route:
  receiver: mail-receiver-dev
  group_by: ['alertname']
  group_wait: 3s
  group_interval: 5s
  repeat_interval: 1h
  # All alerts that do not match the following child routes
  # will remain at the root node and be dispatched to 'default-receiver'.
  routes:
  - receiver: 'mail-pre-dev'
    group_wait: 10s
    match_re: 
    - job = .*pre-dev.*
  - receiver: 'mail-dev'
    group_wait: 10s
    match_re:
    - job = .*dev.*

receivers:
- name: 'mail-dev'
  email_configs:
  - to: '[email protected]'
    send_resolved: true
- name: 'mail-pre-dev'
  email_configs:
  - to: '[email protected]'
    send_resolved: true

I am using the below link as a reference: reference

Testing config file link testscript for using above link: {service="foo-service",severity="critical",job="my-job-sample-service-dev"}

So the question is, how to send an alert to a different channel by using regex for the job title? At the moment when I test all the alert goes to pre-dev.

Solution

  • Change the following:

    match_re: 
- job = .*pre-dev.*

    To:

    matchers: 
- job =~ ".*pre-dev.*"

    Note:

    "match_re" is deprecated and must be replaced by "matchers", but if you want to use it, the correct syntax is:

    match_re:
- job: ".*pre-dev.*"