Search code examples
grafanapromql

Grafana - Limit the amount of shown time series

In Grafana, my datasource is a prometheus database and I am querying using the following query:

topk(10, sum by (pod) (rate(container_cpu_usage_seconds_total{pod=~"^$pod$"}[5m])))

The problem is, that due to the rate and the function container_cpu_usage_seconds_total, I am not receiving the top 10 entries. I am receiving the top 10 entries at a given point of time, which results in many time series displayed in the panel (see the picture below).

GRAFANA QUERY PICTURE(censored)

topk=10, GRAFANA QUERY PICTURE(censored)

Question: Is there a functionality in Grafana to limit the amount of time series show in the panel without using a promQL function like topk(...) ?

The expected outcome would be a panel limited to 10 time series, which come from the given query at the top of the post. I am using Grafana 9.4.7.

Solution

  • Is there a functionality in Grafana to limit the amount of time series show in the panel without using a promQL function like topk(...)?

    I don't think so.

    But you can use topk in a slightly more elaborate query.

    sum by (pod) (rate(container_cpu_usage_seconds_total{pod=~"$pod"}[5m]))
  and topk(10, sum by (pod) (increase(container_cpu_usage_seconds_total{pod=~"$pod"}[$__range]@end())))

    will be showing top 10 pods by total CPU usage over selected for dashboard time range.

    Also, if you are using this in Grafana, consider using $__interval inside of rate to automatically account for scale change on range of time range selected for dashboard.