In the question How to avoid "vector cannot contain metrics with the same labelset" error when plotting rate() from several metrics (same labelset, different names) it was solved, how to build several Prometheus metrics on the same plot in Grafana (and even rate() derivative of the metrics) if we'd like to plot several metrics with common prefix on the same plot.
Now I'd like to plot value of one metrics divided by value of another metrics, so I expect the below PromQL query working:
label_replace({__name__=~"camel_proxy.*sum"},"name_label","$1","__name__", "(.+)")/label_replace({__name__=~"camel_proxy.*count"},"name_label","$1","__name__", "(.+)")
to give me a bunch of graphs where the corresponding *.sum would be divided by corresponding *.count metrics and all the lines would be shown.
However, I just get the empty array from Prometheus. How I can work this around?
Prometheus performs binary operations over pairs of time series with the same set of labels on the left and right side of the binary operator. See vector matching rules for details. In your case the left-side time series contain label_name labels ending with count, while right-side series contain label_name labels ending with sum. Prometheus cannot find series pairs with identical labelsets, so it returns an empty result. The solution is to strip count and sum suffixes from label_name values, so left-side and right-side time series get identical pairs of label_name values:
label_replace(
{__name__=~"camel_proxy.*sum"},"name_label","$1","__name__", "(.+)sum"
) /
label_replace(
{__name__=~"camel_proxy.*count"},"name_label","$1","__name__", "(.+)count"
)