`MethodHandle` slower than Reflection when accessing primitive

Question

I would like to call a method via reflection in the most performant way possible.

The method returns a primitive long.

I've implemented this using both reflection and MethodHandles.

I was expecting MethodHandle to be faster because:

• That's one of the benefits of MethodHandles
• It avoid boxing/unboxing as you experience with reflection

But in all of my benchmarks, MethodHandles are slower (~2-5%-ish).

Take the following JMH benchmark:

import org.openjdk.jmh.annotations.*;

import java.lang.invoke.MethodHandle;
import java.lang.invoke.MethodHandles;
import java.lang.reflect.Method;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Warmup(iterations = 1, time = 100, timeUnit = TimeUnit.MILLISECONDS)
@Measurement(iterations = 5, time = 10, timeUnit = TimeUnit.SECONDS)
@State(Scope.Benchmark)
public class AccessorBenchmark {
    private POJO source;

    private ReflectionAccessor reflectionAccessor;
    private MethodHandleAccessor methodHandleAccessor;

    @Setup
    public void setup() throws ReflectiveOperationException {
        source = new POJO();

        final Method method = source.getClass().getDeclaredMethod("longMethod");

        reflectionAccessor = new ReflectionAccessor(method);
        methodHandleAccessor = new MethodHandleAccessor(method);
    }

    @Benchmark
    public long reflectionAccessor() throws ReflectiveOperationException {
        return reflectionAccessor.get(source);
    }

    @Benchmark
    public long methodHandleAccessor() throws Throwable {
        return methodHandleAccessor.get(source);
    }

    public class ReflectionAccessor {
        private final Object[] EMPTY_ARGS = new Object[0];
        private final Method method;

        public ReflectionAccessor(final Method method) {
            this.method = method;
        }

        public long get(final Object source) throws ReflectiveOperationException {
            return ((Number) method.invoke(source, EMPTY_ARGS)).longValue();
        }
    }

    public class MethodHandleAccessor {
        private MethodHandle methodHandle;

        public MethodHandleAccessor(final Method method) throws ReflectiveOperationException {
            methodHandle = MethodHandles.lookup().unreflect(method);
            methodHandle = methodHandle
                    .asType(methodHandle.type().changeReturnType(long.class).changeParameterType(0, Object.class));
        }

        public long get(final Object source) throws Throwable {
            return (long) methodHandle.invokeExact(source);
        }
    }

    public class POJO {
        // Chose a value outside of the autoboxing cache range
        private final AtomicLong counter = new AtomicLong(Long.MAX_VALUE);

        /** Some dummy method that returns different values in consistent amounts of time */
        public long longMethod() {
            return counter.addAndGet(-1);
        }
    }
}

The following result is returned with Java 17:

Benchmark                                Mode  Cnt     Score    Error   Units
AccessorBenchmark.methodHandleAccessor   avgt   25     4.204 ±  0.546   ns/op
AccessorBenchmark.reflectionAccessor     avgt   25     4.123 ±  0.040   ns/op

Any ideas?

Answer 1

Take a closer look at the Error column in your results:

Score    Error   Units
4.204 ±  0.546   ns/op

It's incorrect to make conclusion about 2-5% performance difference when the measurement error is as high as 13%.

The cause of the high error value is apparently the lack of proper warmup:

@Warmup(iterations = 1, time = 100, timeUnit = TimeUnit.MILLISECONDS)

1 iteration of 100ms is not enough for the benchmark to reach a steady state. Run at least 5 warmup iterations of 1 second each and pay attention to the results of each individual measurement to make sure they do not vary much between iterations.

That's what the same benchmark (with proper warmup) shows on my laptop. MethodHandles here are expectedly faster:

Benchmark                               Mode  Cnt  Score   Error  Units
AccessorBenchmark.methodHandleAccessor  avgt   30  5.946 ± 0.054  ns/op
AccessorBenchmark.reflectionAccessor    avgt   30  6.379 ± 0.069  ns/op