Search code examples
prometheuspromql

Find duplicate labels in prometheus


we have multiple servers, with multiple customers on it. Sometimetimes we copy a customer from one server to another and then point the DNS to the new server. Now, the problem is that the old server still reports some prometheus metrics for that customer, which results in duplicated datasets and bad data.

To clean the data, we need a way to identify those duplicated datasets.

Example Data:

orders(customer="a", server="1") = 200
orders(customer="b", server="1") = 299
orders(customer="c", server="2") = 10
orders(customer="a", server="2") = 10

What we are looking for is some PromQL to give us the following output:

a: 2
b: 1
c: 1

Note that I am not interested in the values here - just in the number of unique datapoints.

Any idea?


Solution

  • Use count and group by (using the by keyword) the label you want to sum, in you case customer

    count(orders) by (customer)
    

    Take a look at this live example