JMeter benchmark JSON vs MessagePack

Question

I want using JMeter to benchmark server to server communication (Java Spring) with different data serialization format than JSON

Article Why not JSON? is suggesting MessagePack

MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it's faster and smaller.

Can I use JMeter for benchmark sending JSON messages vs MessagePack and what can it compare? Can I check the time for getting request on receiver side or also time to prepare and send request on sender side? Or there are other considerations/known issues that prevent either?

Answer 1

You can use JMeter for literally anything, in case of MessagePack you can go for MessagePack Serializer for Java - this will allow you creating binary request payloads on JMeter side i.e. using JSR223 PreProcessor and Groovy language (it's 99.9% Java-compatible so example code will work just fine)

With regards to metrics, they should be the same as usually, your main targets should be:

1. Response time (lower is better)
2. Throughut - number of requests per unit of time (higher is better)

Given JSON and MessagePack are served by the same host network-specific metrics like Latency and Connect Time and be ignored. Check out JMeter Glossary for main metrics listed and explained.

I would also pay attention to server-side metrics like CPU or Memory usage as potentially deserialising binary data and serialising it back can be more resources intensive so my expectation is that MessagePack implementation will have larger footprint. You can use i.e. SSHMon Listener or JMeter PerfMon Plugin in order to check your system under test resources usage while your test is running - this way you will be able to correlate increasing load with increasing resources consumption.