Search code examples
histogramgrafanaresponse-time

Normalize Grafana Histogram response time

I got this histogram in grafana, it represent the distribution of response time of my application. My Histogram

How can I normalize it? Is there a function on Grafana that allows me to do this?

i have multiple query for every percentiles (50th, 90th, 95th,99th)

they are structured as follows:

(histogram_quantile(0.50, sum(irate(istio_request_duration_milliseconds_bucket{reporter="destination",destination_workload=~"$destination_workload"}[1m])) by (destination_workload, destination_workload_namespace, le)) / 1000) or histogram_quantile(0.50, sum(irate(istio_request_duration_seconds_bucket{reporter="destination",destination_workload=~"$destination_workload"}[1m])) by (destination_workload, destination_workload_namespace, le))
Solution

  • I'm afraid you'll need manually normalize your histograms with the following query:

    (
    histogram_quantile(0.50, sum(irate(istio_request_duration_milliseconds_bucket{reporter="destination",destination_workload=~"$destination_workload"}[1m]))
        by (destination_workload, destination_workload_namespace, le)) 
    / 1000
    / on() group_left() sum(histogram_quantile(0.50, sum(irate(istio_request_duration_milliseconds_bucket{reporter="destination",destination_workload=~"$destination_workload"}[1m]))
                            by (destination_workload, destination_workload_namespace, le)))
)
or 
    histogram_quantile(0.50, sum(irate(istio_request_duration_seconds_bucket{reporter="destination",destination_workload=~"$destination_workload"}[1m])) 
        by (destination_workload, destination_workload_namespace, le))
    / on() group_left() sum(histogram_quantile(0.50, sum(irate(istio_request_duration_seconds_bucket{reporter="destination",destination_workload=~"$destination_workload"}[1m]))
                            by (destination_workload, destination_workload_namespace, le)))

    Basically, in this query every item is divided by sum of all items.