Search code examples
prometheuspromqlprometheus-alertmanager

join prometheus queries while keeping data from the left side


I want to build promql query that joins two vectors: one with some metrics and other is informational. The caveat is that info vector doesn't have all the information for "joining label", but I don't want to miss the data from metrics vector.

For example: kafka_consumergroup_lag is the metrics vector with consumergroup label, and catalog_entities_info is the gauge info vector with consumergroup and team label and 1 gauge value for every entry. I want to add team label to kafka_consumergroup_lag. But catalog_entities_info doesn't have all consumergroup and in this case I want to add empty team (or maybe some placeholder, or even not adding anything at all) into resulting vector.

So far I tried:

(sum(kafka_consumergroup_lag[1d]) by (consumergroup) >= 100)
* on(consumergroup) group_left(team)
catalog_entities_info

but this query will filter out lagging consumergroups which are absent in catalog_entities_info.


Solution

  • For now the only solution I have is with query duplication:

    (
      (sum(kafka_consumergroup_lag[1d]) by (consumergroup) >= 100)
      * on(consumergroup) group_left(team)
      catalog_entities_info
    )
    or ignoring(team) (sum(kafka_consumergroup_lag[1d]) by (consumergroup) >= 100)
    

    First half will add team label where possible, and second or half will add missed data (important to merge vectors while ignoring added team label, otherwise we will have duplicates).