How to secure a Java application when dynamic class loading?

I've got a Java/Scala application which performs disk IO and also dynamically loads classes submitted by users. The dynamic classes run under their own ClassLoaders.

What security issues should be considered when loading these classes to ensure that they do not have permission to delete on-disk files or perform anything malicious and are just performing in-memory operations ?

What I've done so far

Regex checks to remove java.io.*, java.nio.*, java.lang.reflect & java.lang.Runtime.* related imports. I'm trying to see if these imports can be removed from the classpath directly instead of doing the regex.
Timeout set for each execution of the dynamically loaded class's instance.
Written test loading 1000s of classes dynamically loaded to check for PermGen issues and there were no memory leaks.

Another option would be to run them in their own JVM process and setting filesystem (Linux) level permissions for each process but this does not fit well in my solution because I need them to share thread-safe in-memory objects.

Can someone please suggest what else should be considered and if I'm on the right track ?

Solution

As @Kayaman said, you should try to use SecurityManager handle these dynamic classes permissions, There is a minimal example to demonstrate this, maybe it's helpful for you.

1.Create Dynamic Classes policy file, this will be used to limit permissions, example, my.policy:

grant {
   permission java.io.FilePermission "*", "read";
};

the above policy will enable read file for any files.

2.Create the custom policy file, that can be used to handle dynamic class with permission validation rules.

class MyPolicy extends Policy {
    //custom classes with policy mapping
    private final Map<String, Policy> plugins;

    MyPolicy(Map<String, Policy> plugins) {
        this.plugins = plugins;
    }

    @Override
    public boolean implies(ProtectionDomain domain, Permission permission) {
        CodeSource codeSource = domain.getCodeSource();
        if (codeSource == null) {
            return false;
        }

        URL location = codeSource.getLocation();
        if (location != null) {
            //get the custom plugin policy rules and validate the permissions
            Policy plugin = this.plugins.get(location.getFile());
            if (plugin != null) {
                return plugin.implies(domain, permission);
            }
        }
        return defaultSystemPermissions().implies(permission);
    }
    private PermissionCollection defaultSystemPermissions() {
        Permissions permissions = new Permissions();
        permissions.add(new AllPermission()); // this will set the application default permissions, in there we enable all 
        return permissions;
    }
}

In the above code, will validate the permission of dynamic classes, if lack of correspond permission, in the runtime, it will throw:

java.security.AccessControlException: access denied ("java.io.FilePermission" "test.txt" "read")

also in there enabled all permission for the default application, maybe should consider more about it in real scenario.

3.setPolicy and install SecurityManager for your dynamic policies.

        // load the dynamic classes
        URL pluginClass = new File("./myplugin").toURI().toURL();
        // read my plugin security policy
        URIParameter myPolicyPath = new URIParameter(MyClass.class.getResource("/my.policy").toURI());
        Policy policy = Policy.getInstance("JavaPolicy", myPolicyPath);
        MyPolicy myPolicy = new MyPolicy(ImmutableMap.of(pluginClass.getPath(), policy));
        Policy.setPolicy(myPolicy);
        // install the security manager
        System.setSecurityManager(new SecurityManager());

4.Full example: TestClass:

public class TestClass {
    public void foobar() throws IOException {
        Path path = Paths.get("test.txt");
        String lines = Files.readAllLines(path).stream().collect(Collectors.joining(","));

        System.out.println(lines);
    }
}

Runner:

public static void main(String[] args) throws Exception{
    // create a new url class loader, this can be used to load a jar or classes directory
    URL pluginClass = new File("./myplugin").toURI().toURL();
    URLClassLoader urlClassLoader = new URLClassLoader(new URL[]{pluginClass}, MyClass.class.getClassLoader());

    // load a dynamic TestClass class
    Class loadedMyClass = urlClassLoader.loadClass("TestClass");

    // read my plugin security policy
    URIParameter myPolicyPath = new URIParameter(MyClass.class.getResource("/my.policy").toURI());
    Policy policy = Policy.getInstance("JavaPolicy", myPolicyPath);
    MyPolicy myPolicy = new MyPolicy(ImmutableMap.of(pluginClass.getPath(), policy));
    Policy.setPolicy(myPolicy);
    // install the security manager
    System.setSecurityManager(new SecurityManager());


    System.out.println("Loaded class: " + loadedMyClass.getName());

    Object myClassObject = loadedMyClass.getConstructor().newInstance();
    Method method = loadedMyClass.getMethod("foobar");
    System.out.println("Invoked method: " + method.getName());
    method.invoke(myClassObject);

}

Reference:

https://docs.oracle.com/javase/tutorial/essential/environment/security.html