Search code examples
prometheus

prometheus recording rule naming guideline when too many labels are exposed


Let us suppose we have recording rule with the expr exposing labels from a,b,c,....z. Is it recommended to create the recording rule with name a_b_c_..............._z:istio_requests:rate5m? Is there a guideline arround the number of labels to be placed in level section of the rule name. (prometheus docs recommended format level:metric:operations)


Solution

  • That's a good question, and I believe one for which no firm answer exists for really long label sets. The naming conventions at https://prometheus.io/docs/practices/rules/#recording-rules were originally inspired by what Google did internally with their Borgmon monitoring system (which was the inspiration for Prometheus), but there were rarely more than a few labels on metrics in Google at the time. So usually aggregated metrics would not have many labels left on them other than e.g. "job" and maybe one or two others. Overall, the recording rule naming conventions are only a loosely followed suggestion in the Prometheus community, so I would do whatever works best for you and your team. In the end, the goal is just that you have a decent idea of the meaning of a metric when you look at its name, and with very long label sets you'll have to strike a balance between the clarity and a too long prefix. If there are for example some informational labels which you never aggregate away anyway at any aggregation level, those could be a good candidate for omission.