I am evaluating a product which works by instrumenting bytecode. I launch my application via the provided agent which instruments bytecode, profiles and generates reports about my application.
What would be the best way to determine the performance penalty imposed by running the agent? Would it be enough if I were to capture latency of a few of operation in my app via JMH with and without the agent? Also, is there a baseline expected drop in performance by using an agent which does bytecode instrumentation?
Thanks
You can download an existing Java performance benchmark like SPECjvm2008 and run it with/without your agent. I wouldn't only write a microbenchmark sourced from the application you are monitoring because that might not highlight the bottlenecks of various operations and instrumentation techniques used by this product(method/memory/system).
The baseline that is advertised for a typical agent is 5%, which is a number I would take with a huge grain of salt.