I'm trying to measure the impact of a JVM agent on performance, to make sure it won't invalidate the tests we're trying to run (and maybe make a case for taking some samples from prod). This case is a set of BTrace scripts which will run during automated load testing, but the issue might be generic to any agent.
To run the benchmarks, I've set up a small JMH project and am attaching the agent as:
java -javaagent:/home/ssube/btrace/build/btrace-agent.jar=scriptdir=/home/ssube/btrace/scripts/,port=0 -jar benchmarks.jar
Doing so causes the following error to appear each time the JVM is forked by JMH:
# Run progress: 0.00% complete, ETA 00:02:00
# Fork: 1 of 1
Exception in thread "main" java.lang.IllegalArgumentException: org.openjdk.jmh.runner.options.CommandLineOptions; local class incompatible: stream classdesc serialVersionUID = 8906142321598115825, local class serialVersionUID = 7529911323947566771
at org.openjdk.jmh.runner.ForkedMain.main(ForkedMain.java:72)
<binary link had failed, forked VM corrupted the stream? Use EXTRA verbose to print exception>
<forked VM failed with exit code 1>
<stdout last='10 lines'>
</stdout>
<stderr last='10 lines'>
Exception in thread "main" java.lang.IllegalArgumentException: org.openjdk.jmh.runner.options.CommandLineOptions; local class incompatible: stream classdesc serialVersionUID = 8906142321598115825, local class serialVersionUID = 7529911323947566771
at org.openjdk.jmh.runner.ForkedMain.main(ForkedMain.java:72)
</stderr>
# VM invoker: /usr/java/jdk1.8.0_11/jre/bin/java
# VM options: -javaagent:/home/ssube/btrace/build/btrace-agent.jar=scriptdir=/home/ssube/btrace/scripts/,port=0
# Warmup: 20 iterations, 1 s each
# Measurement: 20 iterations, 1 s each
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: com.stackoverflow.questions.ShaderBench.testProcessProc
None of my classes are serializable or have a serialVersionUID. The JMH benchmark works without the BTrace agent attached, and the agent and scripts work on the code without JMH.
(How) can you attach a javaagent to a set of JMH benchmarks and capture performance differences caused by the agent?
JMH maintainer here. From maintainer's perspective, it feels like the JMH fault for not supplying SUIDs for its own internal classes, so it can be resistant to benign instrumentation. This is fixed now with 3d44d68e45be:
changeset: 960:3d44d68e45be
tag: tip
user: shade
date: Sat Aug 16 15:00:18 2014 +0400
summary: Apply SUIDs for all Serializable classes: this protects from the benign instrumentation.
Please try with the bleeding edge JMH, and see if it solves the issue for you. Note that providing SUIDs does not protect us from blatant changes in the serialization form the Java agent can make (e.g. injecting the non-transient field), so if BTrace is (mis)behaving like that, we are in trouble.
If the JMH change above does not work, please send some minimal reproducible scenario to jmh-dev mailing list, we would see what can be done to mitigate the issue.
Otherwise, look for excluding org.openjdk.jmh.* classes from instrumentation.