Search code examples
prometheusprometheus-alertmanager

Prometheus Alertmanager config


I have below config to send email alerts from my prometheus based on matching label.

global:
  smtp_from: '[email protected]'
  smtp_smarthost: '1.1.1.1:25'
  smtp_require_tls: false
templates:
- '/etc/alertmanager/default.tmpl'
route:
  receiver: 'default-receiver'
  group_by: [cluster, severity]
  group_wait: 10s
  group_interval: 5m
  repeat_interval: 1h
inhibit_rules:
- source_match:
    severity: critical
  target_match:
    severity: warning
  routes:
  - receiver: 'default-receiver'
  email_configs:
  - to: "[email protected],[email protected],[email protected],[email protected]"
    send_resolved: true
  - receiver: 'datalake-receiver'
  email_configs:
  - to: "[email protected],[email protected],[email protected]"
    match_re:
      kubernetes_namespace: datalake-aws-ec2|datalake-piaas-instances
    send_resolved: true
  - receiver: 'bundling-receiver'
  email_configs:
  - to: "[email protected],[email protected],[email protected],[email protected]"
    match_re:
      kubernetes_namespace: bundling-prod-axa-sg|rabbitmq
    send_resolved: true

I deployed this config but my container is not coming up. Can someone help me verify the syntax and suggest what's wrong in above config. I tried all possible combination but no luck.


Solution

  • There're a lot of issues related to YAML identation and Alertmanager configuration format. The following configuration seems to be working as expected:

    global:
      smtp_from: '[email protected]'
      smtp_smarthost: '1.1.1.1:25'
      smtp_require_tls: false
    
    templates:
    - '/etc/alertmanager/default.tmpl'
    
    route:
      receiver: 'default-receiver'
      group_by: [cluster, severity]
      group_wait: 10s
      group_interval: 5m
      repeat_interval: 1h
    
      routes:
        - receiver: 'datalake-receiver'
          match_re:
            kubernetes_namespace: 'datalake-aws-ec2|datalake-piaas-instances'
        - receiver: 'bundling-receiver'
          match_re:
            kubernetes_namespace: 'bundling-prod-axa-sg|rabbitmq'
    
    receivers:
      - name: 'default-receiver'
        email_configs:
          - to: '[email protected]'
            send_resolved: true
          - to: '[email protected]'
            send_resolved: true
    
      - name: 'datalake-receiver'
        email_configs:
          - to: '[email protected]'
            send_resolved: true
          - to: '[email protected]'
            send_resolved: true
    
      - name: 'bundling-receiver'
        email_configs:
          - to: '[email protected]'
            send_resolved: true
          - to: '[email protected]'
            send_resolved: true
    
    inhibit_rules:
      - source_match:
          severity: critical
        target_match:
          severity: warning
    

    Take a look at the Routing tree editor. It's good to test the Alertmanager configuration.