How to join 2 query results prometheus

Question

I'm trying to query a Prometheus database to determine how many customers have recorded data for one metric with a specific label filter, but not another metric with a different label filter. I.e. all the customer_id's that show up in

sum(usage{usage_type="type_b"}) by (customer_id)

but not in

count(service_plan{plan_type=~".*plan_b.*"}) by (customer_id)

I could run each and just mash them together outside Prometheus, but I want to do this either in a single query in Prometheus, or with some fancy transformation tricks in Grafana.

Answer 1

You need unless operator - see these docs. The following query should return customer ids, which exist in the first query and are missing in the second query:

sum(usage{usage_type="type_b"}) by (customer_id)
  unless
count(service_plan{plan_type=~".*plan_b.*"}) by (customer_id)