I am using PromQL to get the values of a counter for the last 5m minutes but I would like to get the counters delta value for each value returned. I can query using delta(http_requests[5m])
but it only returns the delta of the first and last element. Is there a way to get all values back as delta values? I've looked through the prometheus functions here but cannot find a method to do this.
http_requests[5m]
{
"data": {
"result": [
{
"metric": {
"__name__": "http_requests",
"app": "cloud_engine",
"app_instance": "instance_01",
},
"values": [
[
"2021-10-26T02:33:07Z",
"36446"
],
[
"2021-10-26T02:34:07Z",
"36447"
],
[
"2021-10-26T02:35:07Z",
"36448"
],
[
"2021-10-26T02:36:07Z",
"36450"
],
[
"2021-10-26T02:37:07Z",
"36450"
]
]
}
],
"resultType": "matrix"
},
"status": "success"
}
Desired result showing deltas for each value:
"values": [
[
"2021-10-26T02:33:07Z",
"0"
],
[
"2021-10-26T02:34:07Z",
"1"
],
[
"2021-10-26T02:35:07Z",
"1"
],
[
"2021-10-26T02:36:07Z",
"2"
],
[
"2021-10-26T02:37:07Z",
"0"
]
If you know the interval between samples (e.g. one minute in the example above), then the following subquery can be used:
delta(http_requests[1m])[5m:1m]
This instant query should return the last 5 deltas on 1-minute interval. Unfortunately, it is likely Prometheus will return empty results, since it needs at least two samples on the lookbehind window in square brackets for delta
calculations, e.g. it doesn't take into account the difference between the last point before the lookbehind window and the first point in the lookbehind window. So the query must be rewritten to:
delta(http_requests[2m])[5m:1m]
Unfortumatele this query may give unexpected results because of the same issue - Prometheus skips the delta between the last sample before the lookbehind window and the first sample in the lookbehind window [2m]
.
P.S. An alternative solution is to try the first query in VictoriaMetrics. It takes into account the delta mentioned above, so it must return the expected results. See more details at MetricsQL docs. Take a look also at rollup_delta function - it calculates deltas between consecutive samples on the given lookbehind window and then returns max
, min
and avg
values from the calculated deltas.