CORBA + Tomcat 6 + Webapp classloader leak

EDIT:

I found classloader leak in my webapplication. It boils down to 3rd party library initializing CORBA via JNDI's COS naming service and not exposing a call to cleanly shutdown JNDI's context. This leaves some CORBA related threads and other resources referencing my webapp classloader and preventing it from being garbage collected. This results in OutOfMemory Error: PermGen after few redeploys/reloads.

For now I increased the PermGen memory in JVM and it makes the intervals between server crashes longer. This obviously is not a fix but a workaround (and a poor one for that matter).

I guess my question is is there any way I can cleanly shutdown JNDI context without holding reference to it. My instincts tell me no, but maybe I don't know about some magic feature of JNDI that would allow me to get hold of that context.

So the way the 3rd party library initializes CORBA objects is something along this lines (exception handling and other details omited for brevity):

private CorbaObjectAggregate initCorba() {
    InitialContext ctx = null;
    CorbaObjectAggregate corbaObjects = new CorbaObjectAggregate();
    ORB orb = null;
    Properties env = getContextEnvironment();
    String[] args = null;
    orb = ORB.init(args, null);
    env.put("java.naming.corba.orb", orb);
    ctx = new InitialContext(env);
    //a bunch of object lookups follow
    corbaObjects.someCorbaObjectReference = (SomeCorbaObjectClass) ctx.lookup("somePaht");
    return corbaObjects;
}

So the reference to ctx is gone after that method finishes executing...

I tried stopping the threads manually but it didn't fix the leak. I guess there are some other corba resources holding onto classloader. I suppose I could try to hunt them down in some cleanup method and free the classloader this way, but I was hoping for some cleaner solution.

Just for clarity, the 3rd party library is closed source and I can't really change it. It's also not viable option to get support form the company behind it.

Solution

That is going to be tricky to fix. There are likely to be two issues: - classes loaded from the JAR holding a reference to the web-app class loader - threads started by that process having the web-app class loader as their context class-loader.

Something along the following lines should help:

Move the JAR to $CATALINA_BASE/lib. That will mean that the classes are loaded by the common class loader. The down side is that they are also visible to and shared by every web application.
Find out where in your application the initialisation is triggered. Before that code executes, set the thread context class loader to the system class loader (or the parent of the current (web-app) class loader) and reset the thread context class loader after the init call. That should mean any threads created do not have the web-app class loader as the context class loader.

If threads get created at other points in time, fixing this could get very tricky, very quickly.

For some background that may help understand what is going on, see: http://people.apache.org/~markt/presentations/2010-11-04-Memory-Leaks-60mins.pdf

To see the sort of thing Tomcat does internally to work-around these issues see: http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/catalina/core/JreMemoryLeakPreventionListener.java?view=annotate