Search code examples
prometheuspromqlprometheus-alertmanagerprometheus-operatorprometheus-node-exporter

How can I remove a label and then rename another one to replace it?


I have two different metrics that I am trying to combine in a PromQL expression to write an alert. One of the metrics, say is_up represents if a node is up. The other metric is_active represents if something on that node is running.

Each of them have labels that can be used to compare them, but the labels are called different things on each. So on is_up we have label_a="node_id" and on is_active we have label_b="node_id" where the node_ids are the same.

If the label names were the same, e.g. label, I could write a query that looks like:

count(is_up) by (label) unless count(is_active) by (label)

But this isn't possible. I can't relabel it in a global config as there is already a label on is_up that is called label_b but it has a different format :/

So my current thinking is to (using PromQL, so at evaluation time) remove the label_b from is_up and rename the label label_a to label_b

I saw this answer previously about this topic: https://stackoverflow.com/a/60894489/16546473 but I'm not sure that it applies to me as it's just talking about renaming a label, but I need to

  1. remove label_b from is_up, then
  2. rename label_a in is_up to label_b so that it can be directly compared to label_b in is_active

Let me know if this makes sense


Solution

  • I managed it like this:

    sum without (label_a) (label_replace(sum(is_up) without (label_b), "label_b", "$1", "label_a", "(.*)"))