Let's say we have the metric created with the following time epoch values (each value collected after the previous one, so the instances x and y are not available simultaneously )
created{instance:'x'} 1567000047
created{instance:'x'} 1567000047
created{instance:'x'} 1567000047
created{instance:'y'} 1567000050
created{instance:'y'} 1567000050
created{instance:'y'} 1567000050
Value of label instance is dynamic.
How can I query the data using PromQL to detect that at the time epoch value 1567000047 an instance was created and that at 1567000050 another instance was created?
I would like to plot afterwards the values in Grafana to see when new instances are created. Or at least how can I sum how many instances were created in the last 3 hours (using the existing values)?
I can't sum the values as I will get a straight line. As far as I've seen you can't detect changes from missing to something, only from values like 0, for example.
As I read your request, you want to know the number of time the created metrics changes disregarding the instance label.
Your first issue is about disregarding the instance label because, in order to compute the number of changes over time, you would need to have a function that transform a vector into a vector. This doesn't exists prior to version 2.7 of Prometheus. If you have an older version of prometheus, you'll have to use an adhoc recoding rule.
I'll suppose you have a recent Prometheus, install and that you can use sub-queries. The following should provide a single metric vector giving you the time of the last creation (over the last 3h):
(max(created))[3h:]
Once you get that, you can use the changes() function to compute the number of changes. Add one to account for the first creation:
changes((max(created))[3h:]) + 1
That should give you the number of instance created over the last 3 hours.
Note that sub-queries can be heavy on Prometheus and using a recording rule can help.