I try to export Kafka metrics per JMX to Prometheus and display them with Grafana, but I´m struggling to get the Consumer metrics (to be more precise this one:
kafka.consumer:type=ConsumerFetcherManager,name=MaxLag,clientId=([-.\w]+) )
Everytime I try to fetch this Mbean, it doesn´t even show up. I read all the time that I have to "look into the client", or "I´m looking in the broker metrics, but I need the consumer metrics", but nobody does explain how to do this, so I´m asking you guys if you could help me. Is there some kind of configuration, or special JMX Port to get Consumer metrics or something like that?
The pattern for my config file to look for MBeans:
- pattern : kafka.consumer<type=(.+), name=(.+), client-id=(.+)><>(Count|Value)
name: kafka_consumer_$1_$2
Labels:
clientId: "$3"
Also, i need to fetch the Metrics with JMX, because i dont have access to the Kafka server.
I´m using this project as an example: https://github.com/rama-nallamilli/kafka-prometheus-monitoring
The following two things are possible:
A. May be given client already disconnected from Kafka
B. May be this metric is not present on broker. It might be visible in the JVM application which is running the consumer code. I am not sure but here is how you can check:
Restart your consumer application with JMX enabled
Use visual vm to connect to the above jvm
It should show all the available JMX metrics.
If the metrics contain metrics of your choice then you were looking at wrong place (broker). If not then I am wrong.