Search code examples
prometheusgrafanakamon

Understanding range vector selectors

From reading https://prometheus.io/docs/prometheus/latest/querying/basics/ a 'Range Vector Selectors' are defined as :

Range vector literals work like instant vector literals, except that they select a range of samples back from the current instant. Syntactically, a time duration is appended in square brackets ([]) at the end of a vector selector to specify how far back in time values should be fetched for each resulting range vector element.

In this example, we select all the values we have recorded within the last 5 minutes for all time series that have the metric name http_requests_total and a job label set to prometheus: http_requests_total{job="prometheus"}[5m]

I'm failing to understand how these selector's behave. My understanding of the query :

sum(rate(data_total[1h]))

is return the rate of change of the data_total metric using a time during of 1h. Is this correct ? Is each point on the graph the sum(rate( of the previous hour ?

sum(rate(data_total[1h])) :

From the above definition shouldn't 1h of data be displayed ? Instead may of hours of data are displayed : enter image description here

Changing from 1h to 4h :

sum(rate(data_total[4h])) :

renders :

enter image description here

Why has the shape changed when changing from [1h] to [4h] ?

Solution

  • The range vector selector is just an ordinary time series selector (which is confusingly named instant vector selector) with the added lookbehind window in square brackets.

    An ordinary time series selector (aka instant vector selector) selects time series matching the given filter. For example, http_requests_total{path=~"/foo/bar|/baz"} selects time series with the name http_requests_total and the path label containing either /foo/bar or /baz values.

    The corresponding range vector selector with the one hour lookbehind window looks like the following http_requests_total{path=~"/foo/bar|baz"}[1h].

    The range vector selector can be used in the following places:

    • It can be passed to /api/v1/query. In this case the API returns all the raw samples for matching time series on the interval (time-d ... time], where time is the query arg passed to /api/v1/query, while d is the specified lookbehind window in square brackets of range vector selector. See this article for details.

    • It can be passed to e.g. rollup functions. These functions perform calculations over raw samples on the given lookbehind window in square brackets. The calculations are performed independently per each matching time series and per each requested point on the graph. Prometheus datasource in Grafana sends requests to /api/v1/query_range. This API accepts start, end and step query args and calculates N=1+(end-start)/step points per each matching time series at timestamps start, start+step, start+2*step, ..., start+(N-1)*step.

    Let's look how rate(m[d]) is calculated on the start ... end time range with the given step:

    • Prometheus selects all the time series matching m on the given time range (start-d .. end]. Note that the time range starts from start-d instead of start, where d is the provided lookbehind window in square brackets.

    • Then Prometheus calculates the average per-second increase rate over the given lookbehind window d individually per each matching time series per each requested point on the graph.

    See also this answer.