It could be shown (see below a rewritten test case) that method handles for 254-ary methods claim less memory in Java 17 than in Java 11, when compiled and run by its tools.
Since no method-handle- or reflection-related features are advertised in release summaries for Java versions from 11 to 17, I'm curious: what changes have contributed to less memory consumption?
This is a rewritten test case ArityLimits.java:
import java.lang.invoke.MethodHandle;
import java.lang.invoke.MethodHandles.Lookup;
import java.lang.invoke.MethodHandles;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Consumer;
import java.util.function.Supplier;
class ArityLimits
{
public static void main(String[] args) throws Throwable
{
// Pick an implementation with the 1st argument, e.g. core.
final Invocable invocable = (args.length > 0
&& args[0].equalsIgnoreCase("core"))
? new CoreInvoker(ArityLimits::new)
: new HandleInvoker(ArityLimits::new,
MethodHandles.privateLookupIn(
ArityLimits.class,
MethodHandles.lookup()));
// Pick which methods to call with the 2nd argument, e.g.
// $(( 1|2|4|8 )).
final int agenda = (args.length > 1)
? 0xf & Integer.parseInt(args[1])
: (1 | 2 | 4 | 8);
final Map<String, List<?>> arguments = new HashMap<>(8);
if ((agenda & 1) != 0)
arguments.put("passCrunchLongsFix128",
Collections.nCopies(128 - 2 /* (handle, this) */, 0L));
if ((agenda & 2) != 0)
arguments.put("passClassCrunchLongsFix128",
Collections.nCopies(128 - 1 /* (handle) */, 0L));
if ((agenda & 4) != 0)
arguments.put("passCrunchIntsFix255",
Collections.nCopies(255 - 2 /* (handle, this) */, 0));
if ((agenda & 8) != 0)
arguments.put("passClassCrunchIntsFix255",
Collections.nCopies(255 - 1 /* (handle) */, 0));
final Consumer<Method> announcer = ("Linux".equalsIgnoreCase(
System.getProperty("os.name", "")))
? method -> System.err.format("\033[7m%s\033[0m%n",
method.getName())
: method -> System.err.println(method.getName());
for (Method method : ArityLimits.class.getDeclaredMethods()) {
final List<?> methodArgs = arguments.get(
method.getName());
if (methodArgs == null)
continue;
invocable.invoke(method, methodArgs);
announcer.accept(method);
}
}
private interface Invocable
{
void invoke(Method method, List<?> args) throws Throwable;
static boolean warnTooManyArgs(Method method, List<?> args)
{
if (method.isVarArgs() || args.size() < 256)
return false;
System.err.println(method.getName()
.concat(": too many arguments"));
return true;
}
}
private static final class CoreInvoker implements Invocable
{
private final Supplier<?> instanceer;
CoreInvoker(Supplier<?> instanceer)
{
this.instanceer = instanceer;
}
public void invoke(Method method, List<?> args)
throws ReflectiveOperationException
{
if (Invocable.warnTooManyArgs(method, args))
return;
method.invoke((Modifier.isStatic(method.getModifiers()))
? null
: instanceer.get(),
args.toArray());
}
}
private static final class HandleInvoker implements Invocable
{
private final Supplier<?> instanceer;
private final Lookup lookup;
HandleInvoker(Supplier<?> instanceer, Lookup lookup)
{
this.instanceer = instanceer;
this.lookup = lookup;
}
public void invoke(Method method, List<?> args) throws Throwable
{
// Here a handle shall check its parameter constraints.
final MethodHandle mh1 = lookup.unreflect(method);
if (Invocable.warnTooManyArgs(method, args))
return;
final MethodHandle mh2 = (Modifier.isStatic(
method.getModifiers()))
? mh1
: mh1.bindTo(instanceer.get());
mh2.invokeWithArguments(args);
}
}
void passCrunchLongsFix128(/* MethodHandle mh, ArityLimits al, */
long _001, long _002, long _003, long _004, long _005,
long _006, long _007, long _008, long _009, long _010,
long _011, long _012, long _013, long _014, long _015,
long _016, long _017, long _018, long _019, long _020,
long _021, long _022, long _023, long _024, long _025,
long _026, long _027, long _028, long _029, long _030,
long _031, long _032, long _033, long _034, long _035,
long _036, long _037, long _038, long _039, long _040,
long _041, long _042, long _043, long _044, long _045,
long _046, long _047, long _048, long _049, long _050,
long _051, long _052, long _053, long _054, long _055,
long _056, long _057, long _058, long _059, long _060,
long _061, long _062, long _063, long _064, long _065,
long _066, long _067, long _068, long _069, long _070,
long _071, long _072, long _073, long _074, long _075,
long _076, long _077, long _078, long _079, long _080,
long _081, long _082, long _083, long _084, long _085,
long _086, long _087, long _088, long _089, long _090,
long _091, long _092, long _093, long _094, long _095,
long _096, long _097, long _098, long _099, long _100,
long _101, long _102, long _103, long _104, long _105,
long _106, long _107, long _108, long _109, long _110,
long _111, long _112, long _113, long _114, long _115,
long _116, long _117, long _118, long _119, long _120,
long _121, long _122, long _123, long _124, long _125,
long _126) { }
static void passClassCrunchLongsFix128(/* MethodHandle mh, */
long _001, long _002, long _003, long _004, long _005,
long _006, long _007, long _008, long _009, long _010,
long _011, long _012, long _013, long _014, long _015,
long _016, long _017, long _018, long _019, long _020,
long _021, long _022, long _023, long _024, long _025,
long _026, long _027, long _028, long _029, long _030,
long _031, long _032, long _033, long _034, long _035,
long _036, long _037, long _038, long _039, long _040,
long _041, long _042, long _043, long _044, long _045,
long _046, long _047, long _048, long _049, long _050,
long _051, long _052, long _053, long _054, long _055,
long _056, long _057, long _058, long _059, long _060,
long _061, long _062, long _063, long _064, long _065,
long _066, long _067, long _068, long _069, long _070,
long _071, long _072, long _073, long _074, long _075,
long _076, long _077, long _078, long _079, long _080,
long _081, long _082, long _083, long _084, long _085,
long _086, long _087, long _088, long _089, long _090,
long _091, long _092, long _093, long _094, long _095,
long _096, long _097, long _098, long _099, long _100,
long _101, long _102, long _103, long _104, long _105,
long _106, long _107, long _108, long _109, long _110,
long _111, long _112, long _113, long _114, long _115,
long _116, long _117, long _118, long _119, long _120,
long _121, long _122, long _123, long _124, long _125,
long _126, long _127) { }
void passCrunchIntsFix255(/* MethodHandle mh, ArityLimits al, */
int _001, int _002, int _003, int _004, int _005,
int _006, int _007, int _008, int _009, int _010,
int _011, int _012, int _013, int _014, int _015,
int _016, int _017, int _018, int _019, int _020,
int _021, int _022, int _023, int _024, int _025,
int _026, int _027, int _028, int _029, int _030,
int _031, int _032, int _033, int _034, int _035,
int _036, int _037, int _038, int _039, int _040,
int _041, int _042, int _043, int _044, int _045,
int _046, int _047, int _048, int _049, int _050,
int _051, int _052, int _053, int _054, int _055,
int _056, int _057, int _058, int _059, int _060,
int _061, int _062, int _063, int _064, int _065,
int _066, int _067, int _068, int _069, int _070,
int _071, int _072, int _073, int _074, int _075,
int _076, int _077, int _078, int _079, int _080,
int _081, int _082, int _083, int _084, int _085,
int _086, int _087, int _088, int _089, int _090,
int _091, int _092, int _093, int _094, int _095,
int _096, int _097, int _098, int _099, int _100,
int _101, int _102, int _103, int _104, int _105,
int _106, int _107, int _108, int _109, int _110,
int _111, int _112, int _113, int _114, int _115,
int _116, int _117, int _118, int _119, int _120,
int _121, int _122, int _123, int _124, int _125,
int _126, int _127, int _128, int _129, int _130,
int _131, int _132, int _133, int _134, int _135,
int _136, int _137, int _138, int _139, int _140,
int _141, int _142, int _143, int _144, int _145,
int _146, int _147, int _148, int _149, int _150,
int _151, int _152, int _153, int _154, int _155,
int _156, int _157, int _158, int _159, int _160,
int _161, int _162, int _163, int _164, int _165,
int _166, int _167, int _168, int _169, int _170,
int _171, int _172, int _173, int _174, int _175,
int _176, int _177, int _178, int _179, int _180,
int _181, int _182, int _183, int _184, int _185,
int _186, int _187, int _188, int _189, int _190,
int _191, int _192, int _193, int _194, int _195,
int _196, int _197, int _198, int _199, int _200,
int _201, int _202, int _203, int _204, int _205,
int _206, int _207, int _208, int _209, int _210,
int _211, int _212, int _213, int _214, int _215,
int _216, int _217, int _218, int _219, int _220,
int _221, int _222, int _223, int _224, int _225,
int _226, int _227, int _228, int _229, int _230,
int _231, int _232, int _233, int _234, int _235,
int _236, int _237, int _238, int _239, int _240,
int _241, int _242, int _243, int _244, int _245,
int _246, int _247, int _248, int _249, int _250,
int _251, int _252, int _253) { }
static void passClassCrunchIntsFix255(/* MethodHandle mh, */
int _001, int _002, int _003, int _004, int _005,
int _006, int _007, int _008, int _009, int _010,
int _011, int _012, int _013, int _014, int _015,
int _016, int _017, int _018, int _019, int _020,
int _021, int _022, int _023, int _024, int _025,
int _026, int _027, int _028, int _029, int _030,
int _031, int _032, int _033, int _034, int _035,
int _036, int _037, int _038, int _039, int _040,
int _041, int _042, int _043, int _044, int _045,
int _046, int _047, int _048, int _049, int _050,
int _051, int _052, int _053, int _054, int _055,
int _056, int _057, int _058, int _059, int _060,
int _061, int _062, int _063, int _064, int _065,
int _066, int _067, int _068, int _069, int _070,
int _071, int _072, int _073, int _074, int _075,
int _076, int _077, int _078, int _079, int _080,
int _081, int _082, int _083, int _084, int _085,
int _086, int _087, int _088, int _089, int _090,
int _091, int _092, int _093, int _094, int _095,
int _096, int _097, int _098, int _099, int _100,
int _101, int _102, int _103, int _104, int _105,
int _106, int _107, int _108, int _109, int _110,
int _111, int _112, int _113, int _114, int _115,
int _116, int _117, int _118, int _119, int _120,
int _121, int _122, int _123, int _124, int _125,
int _126, int _127, int _128, int _129, int _130,
int _131, int _132, int _133, int _134, int _135,
int _136, int _137, int _138, int _139, int _140,
int _141, int _142, int _143, int _144, int _145,
int _146, int _147, int _148, int _149, int _150,
int _151, int _152, int _153, int _154, int _155,
int _156, int _157, int _158, int _159, int _160,
int _161, int _162, int _163, int _164, int _165,
int _166, int _167, int _168, int _169, int _170,
int _171, int _172, int _173, int _174, int _175,
int _176, int _177, int _178, int _179, int _180,
int _181, int _182, int _183, int _184, int _185,
int _186, int _187, int _188, int _189, int _190,
int _191, int _192, int _193, int _194, int _195,
int _196, int _197, int _198, int _199, int _200,
int _201, int _202, int _203, int _204, int _205,
int _206, int _207, int _208, int _209, int _210,
int _211, int _212, int _213, int _214, int _215,
int _216, int _217, int _218, int _219, int _220,
int _221, int _222, int _223, int _224, int _225,
int _226, int _227, int _228, int _229, int _230,
int _231, int _232, int _233, int _234, int _235,
int _236, int _237, int _238, int _239, int _240,
int _241, int _242, int _243, int _244, int _245,
int _246, int _247, int _248, int _249, int _250,
int _251, int _252, int _253, int _254) { }
}
This is an argument file args:
-XX:+UnlockExperimentalVMOptions
-XX:+UseEpsilonGC
-XX:+AlwaysPreTouch
-Xms136m
-Xmx136m
-Xlog:heap*=info,gc=info
Save the files in, for example, a /tmp directory, and start a Java 11 version Docker container:
docker run --entrypoint /bin/sh --interactive \
--rm --tty --volume="/tmp:/tmp" \
--workdir="/tmp" eclipse-temurin:11-jdk-alpine # It may pull in ~200 MiB.
javac -Xdiags:verbose -Xlint ArityLimits.java
java @args ArityLimits handle $(( 1 | 2 | 4 | 8 )) # 91/136 MiB of heap used
java @args ArityLimits handle 8 # 36/136 (see title)
java @args ArityLimits core $(( 1 | 2 | 4 | 8 )) # 1/136
java @args ArityLimits core 8 # 1/136
rm *.class
exit
Now, start a Java 17 version Docker container:
docker run --entrypoint /bin/sh --interactive \
--rm --tty --volume="/tmp:/tmp" \
--workdir="/tmp" eclipse-temurin:17-jdk-alpine # It may pull in ~200 MiB.
javac -Xdiags:verbose -Xlint ArityLimits.java
java @args ArityLimits handle $(( 1 | 2 | 4 | 8 )) # 5/136 MiB of heap used
java @args ArityLimits handle 8 # 3/136 (see title)
java @args ArityLimits core $(( 1 | 2 | 4 | 8 )) # 1/136
java @args ArityLimits core 8 # 1/136
rm *.class
exit
The main reason for the high memory consumption is, of course, the option -XX:+UseEpsilonGC, as turning off the garbage collector implies having all temporary objects in memory instead of being reclaimed.
You can simply make a heap dump at the end of the program to see the dominating objects and use a profiling tool to track the allocations. I found quite a lot StringBuilder instances in Java 11 (170,000) and in turn, byte[] arrays, compared to Java 17 (4,000 StringBuilder instances). One of the source of allocations revealed by async-profiler was jdk.internal.org.objectweb.asm.Type which is a class of the ASM library embedded in Java.
Java 11 uses version 6 of this library whereas Java 17 uses version 8. In-between these versions there is a change documented as
small optimizations in asm.Type
One of the frequently invoked methods in this class is
public String getDescriptor() { StringBuilder buf = new StringBuilder(); getDescriptor(buf); return buf.toString(); }
we don’t need to dig into the actual implementation method to see that there’s a StringBuilder allocated unconditionally. In contrast, the newer implementation looks like
public String getDescriptor() { if (sort == OBJECT) { return valueBuffer.substring(valueBegin - 1, valueEnd + 1); } else if (sort == INTERNAL) { return 'L' + valueBuffer.substring(valueBegin, valueEnd) + ';'; } else { return valueBuffer.substring(valueBegin, valueEnd); } }
which will allocate a StringBuilder only in some cases. This especially applies to your case of having lots int or long parameters, which will end up at the last branch, just calling substring on the method signature to produce a string equal to "I" or "J". In contrast, the old code allocated a StringBuilder instance and a byte[] array of length 16 (the default capacity) in addition to the result string, for each invocation.
There might be more allocations sites, differing between the Java versions, but I think this provides you with the necessary information about where to search and how to search.