Search code examples
javaapache-kafkajmxmbeans

Get kafka topic metrics without client-ID

I am attempting to gather kafka topic metrics using JMX so that I can pack them into an object in java. The Kafka Docs show that the MBean requires a client-id, however we do not have any client-ids set up in our system.

What leads me to believe that it is possible to get topic level metrics without a client id is the fact that our instance of the Kafka Manager service is able to pull out topic metrics. I found the MBean they use "kafka.server:name=MessagesInPerSec,topic=topic,type=BrokerTopicMetrics" and attempted to use that with no results.

My current code is

try {
   JMXServiceURL target = new 
   JMXServiceURL("service:jmx:rmi:///jndi/rmi://KAFKA_URL:9999/jmxrmi");
   JMXConnector connector = JMXConnectorFactory.connect(target);
   MBeanServerConnection remote = connector.getMBeanServerConnection();
   String beanName = "kafka.server:name=MessagesInPerSec,topic=topicName,type=BrokerTopicMetrics";
   ObjectName bean = new ObjectName(beanName);
   MBeanInfo info = remote.getMBeanInfo(bean);
   LOGGER.info(info.getDescription());
   MBeanAttributeInfo[] attributes = info.getAttributes();
   for (MBeanAttributeInfo attr : attributes) {
      LOGGER.info("^C " + attr.getName() + " " + remote.getAttribute(bean,attr.getName()));
   }
   connector.close();
}
catch(Exception e) {
   LOGGER.info(e.getMessage());
}

I have been able to get regular broker level metrics through this method, It is just when I try and get down into the individual topic metrics that I lose output Thank you!

Solution

  • The code you pasted above works for me, it's correctly printing the metrics of the specified topic for that broker. However note that if the broker is not hosting any partitions (or replicas) of the specified topic, you will get InstanceNotFoundException back.

    An easy way to retrieve MBean names is to use jconsole. You can browse the metrics and once you found one you want, you can get its name in the "Operations" item just below "Attributes".

    Otherwise use the queryNames() method to find all available ObjectNames on the MBeanServerConnection:

    Set<ObjectName> names = remote.queryNames(null, null);