Alertmanager failing due to "unmarshal errors: line 19: field matchers not found in type config.plain"

Question

I am trying to add custom alert-routing config to my alertmanager, deployed as a part of kube-prometheus-stack. But prometheus-operator pod, while trying to generate the alertmanager configmap, fails due to the following error:

level=error ts=2021-05-31T06:29:38.883470881Z caller=klog.go:96 component=k8s_client_runtime func=ErrorDepth msg="Sync \"infra-services/prometheus-operator-kube-p-alertmanager\" failed: provision alertmanager configuration: base config from Secret could not be parsed: yaml: unmarshal errors:\n line 19: field matchers not found in type config.plain"

I also validated the same using amtool inside alertmanager container, which gives the same error. Here is my alertmanager.yml file:

global:
  resolve_timeout: 5m
  slack_api_url: https://hooks.slack.com/services/xxxxxx/yyyyy/zzzzzzzzzzz
receivers:
- name: slack-notifications
  slack_configs:
  - channel: '#alerts'
    send_resolved: true
    text: '{{ template "slack.myorg.text" . }}'
- name: blackhole-receiver
route:
  group_by:
  - alertname
  group_interval: 5m
  group_wait: 30s
  receiver: blackhole-receiver
  repeat_interval: 12h
  routes:
  - matchers:
    - severity=~"warning|critical"
    receiver: slack-notifications
templates:
- /etc/alertmanager/config/*.tmpl

I have followed https://prometheus.io/docs/alerting/latest/configuration/ and https://github.com/prometheus/alertmanager/blob/master/doc/examples/simple.yml to write my simple alertmanager config.

Answer 1

Try to change from:

  routes:
  - matchers:
    - severity=~"warning|critical"
    receiver: slack-notifications

To:

  routes:
    - match_re:
        severity: "warning|critical"
      receiver: slack-notifications