Search code examples
prometheusgrafana

Grafana panel showing client request count per HTTP status

I would like to see how many times I get 200, 4xx, 5xx within a selected period of time. The query I managed to write with my limited knowledge is:

sum(increase(http_client_requests_second_count{uri=~"$apiUri"}[1m])) by (uri, method, status)

and the result I got is

grafana_panel

and the result I was hoping to see was 1 for 400 and 18 for CLIENT_ERROR, as seen in Prometheus:

prometheus_query

What would be the correct query?

Thanks in advance...

Solution

  • First of all, you show different data.

    Prometheus' web ui screenshot shows plain metric http_client_requests_second_count which is is the total number of requests made to endpoint in labels (counting from previous reset - most likely restart of application).

    Grafana's screenshot shows result of mentioned query. And this query gets number of requests to server for the last one minute aggregated by uri, method, and status of request/response.

    And if you want to see number of request per HTTP status (for the last minute) you 'll need similar query:

    sum(increase(http_client_requests_second_count{uri=~"$apiUri"}[1m])) by (status)

    Or

    sum(http_client_requests_second_count{uri=~"$apiUri"} - http_client_requests_second_count{uri=~"$apiUri"} offset 1m) by (status)

    if you want integer numbers (and don't mind negatives near restarts of app)

    metric - metric offset 1m counts difference between current value of metric and its value 1 minute ago. It produces integer result (for integer metrics), while increase() could produce float (due to nature of inner working of increase()). But near resets this query will produce negative values (because it's plain subtraction), while increase() will produce correct approximation near resets too.