Search code examples
prometheuspromql

Get the increase for a prometheus metric that has been recorded as a rate

Let's say I have a metric foo_count that get's rolled up before being written to remote storage to reduce cardinality and uses this rule:

- record: rollup:foo_count:1m
  expr: sum(rate(foo_count[1m])) without(pod, nodename, instance)

How do I get the 24h total of rollup:foo_count:1m? For the raw metric, I know I could simply do sum(increase(foo_count)[24h]).

I thought I could use sum(sum_over_time(rollup:foo_count:1m)[24h]), but the results are not close at all.

Any ideas on how to write the correct query to get the total of the rollup over the last 24h?

Solution

  • Okay, I figured it out with avg_over_time:

    sum(avg_over_time(rollup:foo_count:1m[24h])) * 24 * 60 * 60)

    Basically, you average the rate out over the period... then multiple it by the number of seconds in the same period