Search code examples
prometheusprometheus-alertmanager

Prometheus alert rule for metric incrementing every minute


I have a metric that keeps track of error count. I want to have rule wherein there is an alert when the error count is increasing and is > 100 every minute for the last 5 minutes.

 rules:
      - alert: Error count greater than threshold     
        expr: error_count > 100
        for: 5m

The above alert just checks if the error count is > 100 for 5 mins, but I want to check if the error count is increasing by 100 every minute. Is that possible to check?


Solution

  • if u want to see 5 last minute you must be use average in 5 min like this

    avg_over_time(error_count[5m])

    and if u want to see your value in alert response (integration) you must be add value into label like this

            expr: error_count > 100
            for: 5m
            labels:
              severity : critical
              value : "{{ $value }}"