Search code examples
prometheusgrafanapromql

How to deconflict Prometheus joins with overlapping/colliding labels?


Multiple joins of metrics with same labels gets overridden by last group

I am trying to left join from multiple series but having issues with label collisions.

My query is as follows: server_info{} + on(instance) group_left(version) jvm_info{} + on(instance) group_left(version, commit) application_build_info{}

The second "version" overrides the first "version". Is there any way to "relabel" at query-time or something similar to MySQL "version as app_version"?

I could not find anything in the documentation, and my google-fu is failing me.

PS. This is in Grafana so solutions using the built in merge/group-by/joins is acceptable.


Solution

  • label_replace() function solves the problem

    Edit: Thanks to @valyala for explaining the difference between subqueries and functions. Learn something new every day.

    For anyone stumbling over this in the future I managed to get it working using label_replace() function. Couldn't get it to work with without(version) to remove it, so order matters.

    server_info{}
    + on(instance) group_left(version) jvm_info{}
    + on(instance) group_left(app_version, commit) (label_replace(application_build_info{}, "app_version", "$1", "version", "(.*)"))