Monitoring Kafka Topic by Jmeter

Question

I am trying to monitor a Kafka topic through jmeter. I am sending 100 messages to the kafka topic though an external tool and the messages are passing perfectly as I have checked in the consumer window. Now I want to monitor the kafka topic using jmeter. I have looked into using kafkameter and seen some tutorials on load testing using kafkameter. But in all the resources I found online it was shown how to send messages to Kafka topic and then monitor it which I don't want to do. I want to monitor the topic while I send messages to that topic from external tool. Can anyone help me with this?

Answer 1

I don't know what do you mean by "monitoring", if you want to receive messages from the given Kafka topic - you can use JSR223 Sampler and Kafka Consumer API to read the messages and do whatever you want to do with them.

1. Download the libraries providing Kafka Consumer API to JMeter Classpath
2. Restart JMeter to pick the libraries up

3. Add JSR223 Sampler to your Thread Group and put the code which will establish the connection with the "interesting" Kafka topic, to give you an overall idea here is a simple example code which waits for incoming messages and prints them to the jmeter.log file

   props.put('bootstrap.servers', '192.168.99.100:9092')
   props.put('group.id', 'foo')
   props.put('enable.auto.commit', 'true')
   props.put('auto.commit.interval.ms', '1000')
   props.put('session.timeout.ms', '30000')
   props.put('key.deserializer',
           'org.apache.kafka.common.serialization.StringDeserializer')
   props.put('value.deserializer',
           'org.apache.kafka.common.serialization.StringDeserializer')
   def consumer = new org.apache.kafka.clients.consumer.KafkaConsumer<String, String>(props)
   def topic = 'sometopic'
   
   consumer.subscribe(Arrays.asList(topic))
   log.info('Subscribed to topic ' + topic)
   
   while (true) {
       def records = consumer.poll(100)
       records.each { record ->
           log.info('Received message: ' + record.value())
       }
   }
   

Demo:

More information: Apache Kafka - How to Load Test with JMeter