Creating Lambda function instance

Question

I'm trying to understand lambda expressions and got the following issue. I understand that lambda-expression is compiled to invokedynamic instruction by javac and basic mechanics of indy.

I have the class loader:

public class MyClassLoader extends ClassLoader{
    public Class<?> defineClass(byte[] classData){
        Class<?> cls = defineClass(null, classData, 0, classData.length);
        resolveClass(cls); 
        return cls; //should be ok, resolved before returning
    }
}

Now I want to create a hand-crafted Class dynamically with ASM and use it in LambdaMetafactory to create an instance of my functional interface. Here it is:

@FunctionalInterface
public interface Fnct {
    Object apply(String str);
}

Here is my full application:

public static void main(String[] args) throws Throwable {
    System.out.println(
          generateClassWithStaticMethod().getMethod("apply", String.class)
                 .invoke(null, "test")   
    ); //prints 3 as expected

    MethodHandles.Lookup lookup = MethodHandles.lookup();
    MethodHandle mh = lookup.findStatic(generateClassWithStaticMethod(), "apply", MethodType.methodType(Object.class, String.class));
    Fnct f =  (Fnct) LambdaMetafactory.metafactory(lookup, "apply", MethodType.methodType(Fnct.class),
            mh.type(), mh, mh.type()).getTarget().invokeExact();

    f.apply("test"); //throws java.lang.NoClassDefFoundError: MyTestClass
}

public static Class<?> generateClassWithStaticMethod(){
    ClassWriter classWriter = new ClassWriter(ClassWriter.COMPUTE_MAXS);

    classWriter.visit(V1_8, ACC_PUBLIC + ACC_SUPER, "MyTestClass", null, getInternalName(Object.class), null);

    MethodVisitor mv = classWriter.visitMethod(ACC_PUBLIC + ACC_STATIC, "apply", "(Ljava/lang/String;)Ljava/lang/Object;",null, null);
    mv.visitInsn(ICONST_3);
    mv.visitMethodInsn(INVOKESTATIC, getInternalName(Integer.class), "valueOf", "(I)Ljava/lang/Integer;", false);
    mv.visitInsn(ARETURN);
    mv.visitMaxs(0, 0);
    mv.visitEnd();

    return new MyClassLoader().defineClass(classWriter. toByteArray());
}

So reflective method call succeeds, but creating and invoking instance with LambdaMetafactory fails with NoClassDefFoundError. I tried to create a class in Java with a static Method, and it worked:

public class Fffnct {
    public static Object apply(String str){
        return 3;
    }
}

The only difference in class-files I found is that javac generates:

LineNumberTable:
    line 5: 0

I tried to add it by myself as mv.visitLineNumber(5, new Label()); but unfortunately, it didn't work.

What's wrong with my dynamically generated class?

Answer 1

The crucial part is the MethodHandles.Lookup instance which defines the context the lambda will live in. Since you’ve created it via MethodHandles.lookup() in your main method, it encapsulates a context where the classes defined by your new class loader are not visible. You can change the context via in(Class) but this will change the access modes and cause the LambdaMetaFactory to reject the lookup object. In Java 8, there is no standard way to create a lookup object having private access to another class.

Just for demonstration purposes, we can use Reflection with access override to produce an appropriate lookup object, to show that it will work then:

Class<?> generated = generateClassWithStaticMethod();
MethodHandles.Lookup lookup = MethodHandles.lookup().in(generated);
Field modes = MethodHandles.Lookup.class.getDeclaredField("allowedModes");
modes.setAccessible(true);
modes.set(lookup, -1);
MethodHandle mh = lookup.findStatic(generated, "apply", MethodType.methodType(Object.class, String.class));
Fnct f =  (Fnct) LambdaMetafactory.metafactory(lookup, "apply", MethodType.methodType(Fnct.class),
        mh.type(), mh, mh.type()).getTarget().invokeExact();
Object result = f.apply("test");
System.out.println("result: "+result);

But, as we all know, Reflection with access override is discouraged, it will generate a warning in Java 9, and may break in future versions, as well as other JREs perhaps not even having this field.

On the other hand, Java 9 introduced a new way to get a lookup object, if the current module dependencies do not forbid it:

Class<?> generated = generateClassWithStaticMethod();
MethodHandles.Lookup lookup = MethodHandles.lookup();
lookup = MethodHandles.privateLookupIn(generated, lookup);// Java 9
MethodHandle mh = lookup.findStatic(generated, "apply", MethodType.methodType(Object.class, String.class));
Fnct f =  (Fnct) LambdaMetafactory.metafactory(lookup, "apply", MethodType.methodType(Fnct.class),
        mh.type(), mh, mh.type()).getTarget().invokeExact();
Object result = f.apply("test");
System.out.println("result: "+result);

Another option introduced by Java 9, is to generate a class into your own package instead of a new class loader. Then, it’s accessible to your own class lookup context:

public static void main(String[] args) throws Throwable {
    byte[] code = generateClassWithStaticMethod();
    MethodHandles.Lookup lookup = MethodHandles.lookup();
    Class<?> generated = lookup.defineClass(code);// Java 9
    System.out.println("generated "+generated);
    MethodHandle mh = lookup.findStatic(generated, "apply", MethodType.methodType(Object.class, String.class));
    Fnct f =  (Fnct) LambdaMetafactory.metafactory(lookup, "apply", MethodType.methodType(Fnct.class),
            mh.type(), mh, mh.type()).getTarget().invokeExact();
    Object result = f.apply("test");
    System.out.println("result: "+result);
}

public static byte[] generateClassWithStaticMethod() {
    ClassWriter classWriter = new ClassWriter(ClassWriter.COMPUTE_MAXS);
    classWriter.visit(V1_8, ACC_PUBLIC + ACC_SUPER, "MyTestClass", null, "java/lang/Object", null);
    MethodVisitor mv = classWriter.visitMethod(ACC_PUBLIC + ACC_STATIC, "apply", "(Ljava/lang/String;)Ljava/lang/Object;",null, null);
    mv.visitInsn(ICONST_3);
    mv.visitMethodInsn(INVOKESTATIC, "java/lang/Integer", "valueOf", "(I)Ljava/lang/Integer;", false);
    mv.visitInsn(ARETURN);
    mv.visitMaxs(0, 0);
    mv.visitEnd();
    byte[] byteArray = classWriter.toByteArray();
    return byteArray;
}

If you continue to use a custom class loader, you could utilize the fact that you are doing code generation anyway. So you could generate a method calling MethodHandles.lookup() in your generated class and returning it. Then, invoke it via Reflection and you have hands on a lookup object representing the context of the generated class. On the other hand, you could also insert the instruction to generate the lambda instance right into the generated class itself:

public static void main(String[] args) throws Throwable {
    String staticMethodName = "apply";
    MethodType staticMethodType = MethodType.methodType(Object.class, String.class);
    Class<?> generated = generateClassWithStaticMethod("TestClass", Object.class,
        staticMethodName, staticMethodType, Fnct.class, "apply", staticMethodType);
    MethodHandles.Lookup lookup = MethodHandles.lookup();
    System.out.println("generated "+generated);
    MethodHandle mh = lookup.findStatic(generated, "apply", MethodType.methodType(Fnct.class));
    Fnct f =  (Fnct)mh.invokeExact();
    Object result = f.apply("test");
    System.out.println("result: "+result);
}

public static Class<?> generateClassWithStaticMethod(String clName, Class<?> superClass,
    String methodName, MethodType methodType, Class<?> funcInterface, String funcName, MethodType funcType) {

    Class<?> boxedInt = Integer.class;
    ClassWriter classWriter = new ClassWriter(0);
    classWriter.visit(V1_8, ACC_PUBLIC|ACC_SUPER, clName, null, getInternalName(superClass), null);
    MethodVisitor mv = classWriter.visitMethod(
         ACC_PUBLIC|ACC_STATIC, methodName, methodType.toMethodDescriptorString(), null, null);
    mv.visitInsn(ICONST_3);
    mv.visitMethodInsn(INVOKESTATIC, getInternalName(boxedInt), "valueOf",
        MethodType.methodType(boxedInt, int.class).toMethodDescriptorString(), false);
    mv.visitInsn(ARETURN);
    mv.visitMaxs(1, 1);
    mv.visitEnd();
    String noArgReturnsFunc = MethodType.methodType(funcInterface).toMethodDescriptorString();
    mv = classWriter.visitMethod(ACC_PUBLIC|ACC_STATIC, methodName, noArgReturnsFunc, null, null);
    Type funcTypeASM = Type.getMethodType(funcType.toMethodDescriptorString());
    mv.visitInvokeDynamicInsn(funcName, noArgReturnsFunc, new Handle(H_INVOKESTATIC,
        getInternalName(LambdaMetafactory.class), "metafactory", MethodType.methodType(CallSite.class,
            MethodHandles.Lookup.class, String.class, MethodType.class, MethodType.class,
            MethodHandle.class, MethodType.class).toMethodDescriptorString()), funcTypeASM,
            new Handle(H_INVOKESTATIC, clName, methodName, methodType.toMethodDescriptorString()),
            funcTypeASM
        );
    mv.visitInsn(ARETURN);
    mv.visitMaxs(1, 0);
    mv.visitEnd();
    return new MyClassLoader().defineClass(classWriter.toByteArray());
}

This generates a second static method with the same name but no arguments, returning an instance of the functional interface, generated exactly like a method reference to the first static method, using a single invokedynamic instruction. Of course, that’s merely to demonstrate the logic, as it would be easy to generate a class implementing the interface performing the action within its function method directly, instead of requiring the meta factory to generate a delegating class.