Adding custom labels to jvm related metrics using jmx_exporter

Question

I want to add custom labels "key:my-app" for the jvm.* or process.* related metrics when using prometheus_jmx_exporter with kafka. I couldn't figure out the rules/patterns to be used for the same. Logged Github comment as well on a similar issue, but no response.

The following doesn't add the label "services="my-app" to the generated metrics:

rules:
- pattern : process.cpu.seconds.total
  name: kafka_process_cpu_seconds_total
  labels:
    services: "my-app"

Answer 1

In Prometheus such labels are called target labels, and are applied to on the Prometheus end. Usually this is done with relabel_configs, based on metadata from service discovery.