Search code examples
prometheusgrafanapromql

How to filter prometheus query results based on the maximum value in a time range?

I am monitoring a series of CPU load data, which is aggregated. Similar to:

avg(load_data{}) by (service_name)

Now, I just want to filter out the results that meet the following conditions: In the past n days, the maximum value of one service is greater than 90%.

My purpose is to focus on the service who had a high CPU usage.

How can I do that? I tried max_over_time() but not work.

Solution

  • You need to use subquery, like this:

    max_over_time(avg by (service_name) (load_data)[1d:])

    More info in the Prometheus documentation here.