I have an application that consists of a grid of hazelcast nodes which uses extensive runtime bytecode generation (asm). Specifically I am dynamically building predicate<> Java functions from user entered filter expressions. I would like to store the predicates in a map so that they are available across the cluster without having to be recompiled. Predicates are not the only instance of this. I also have ORM style mapping classes that are generated at runtime which need to be shared across the cluster. These classes are loaded by a custom classloader called DynamiClassLoader.
At first I was unable to store my custom generated classes in Hazelcast IMaps, getting a ClassNotFoundexception. However, I found if tell Hazelcast to use my custom DynamicClassLoader using Config.setClassLoader() then it is able to find my dynamic classes and use them in the local member so I have no issues serializing and deserializing instances of my custom classes in IMaps in the same member.
However, I am still unable to deserialize instances of my predicates which were inserted into the map by a different member. I have enabled UserCodeDeployment and stepped through the code in my debugger to confirm that if it cannot find the class locally then it is hitting UserCodeDeploymentClassLoader.java and on the classNotFoundException it is proceeding to check other members for the classes but it is unable to find them. I haven't been able to discover how exactly this works. it seems to look in an internal map for which member any given class can be found on and it does not find my classes in there. I believe it dispatches an operation to other members then to look for the class, but in this case it looks like my custom classloader isn't used so it's not able to find my custom classes.
How can I make dynamically generated classes work with UserCodeDeployment on Hazelcast? Is there a way I can 'register' my dynamic classes with the member's code service or something like that?
Thankyou, Troy.
I've finally figured this out after extensive debugging. It turns out that the operation in Hazelcast on the target member to find a class calls loadBytecodeFromParent() in ClassDataProvider.java. This looks for a .class file using getResourceAsStream on the classloader:
String resource = className.replace('.', '/').concat(".class");
...
is = parent.getResourceAsStream(resource);
Which basically looks for the class file on the filesystem. Since my dynamic classes are entirely in memory there is no .class file resource for it to find.
I resolved this, by putting a hashmap in my DynamicClassLoader to keep the generated bytecode in and overriding getResourceAsStream to return that bytecode when it's available before looking further. Now it works!