Search code examples
prometheusgrafanapromql

Calculate sum of the Prometheus counter's values even if application restarted

As you know, Prometheus counters reset after application restarts. Is there any way to calculate the sum of the counter's values in Grafana somehow even after restarts? Found old question How to sum prometheus counters when k8s pods restart but might be something new introduced since 2019. enter image description here

Solution

  • There is no need for any trickery: function increase deals with resets automatically:

    Breaks in monotonicity (such as counter resets due to target restarts) are automatically adjusted for.

    So query like increase(metric [30d]) will calculate total change in the counter value over last 30 days.

    Also notice, that results of increase might differ slightly from manually calculated, as there is some extrapolation involved:

    The increase is extrapolated to cover the full time range as specified in the range vector selector, so that it is possible to get a non-integer result even if a counter increases only by integer increments.