Search code examples
javapostgresqltomcatjdbcmemory-leaks

Does Postgres JDBC LazyCleaner cause a classloader memory leak?

I have a Spring Boot web application which is deployed multiple times a day, so a clean undeploy is essential. Tomcat is reporting a memory leak, and MAT points me to

Path to GC Root

I'm using postgresql-42.6.0.jar, and have placed that JAR in the Tomcat 10.1 ./lib directory (as JDBC drivers should be, afaik). It seems to me the web app present at Tomcat startup loads org.postgresql.util.LazyCleaner, which then starts the "PostgreSQL-JDBC-Cleaner" thread. As the thread inherits the AccessControlContext of the ProtectionDomain loaded by the webapp class loader, that class loader will forever have a reference which cannot be gc'ed, ergo a memory leak.

I can't find anything online that others are also having this issue. Am I missing something?

Solution

  • LazyCleaner was introduced with 42.6.0, as an attempt to deal with memory leaks (https://github.com/pgjdbc/pgjdbc/issues/1360 and https://github.com/pgjdbc/pgjdbc/issues/1431). The class uses Java's PhantomReference to find when a "referent" object is gc'ed, and then runs an associated 'CleaningAction'. All this runs in a "PostgreSQL-JDBC-Cleaner" thread. If the thread doesn't have any "referent" objects at all within a TTL (default is 30s), the thread exits. Registering any "referent" object will create a new thread if necessary.

    What parts of postgresql-42.6.0.jar use the LazyCleaner?

    PgConnection.java
cleanable = LazyCleaner.getInstance().register(leakHandle, finalizeAction);

StreamWrapper.java
cleaner = LazyCleaner.getInstance().register(leakHandle, tempFileHolder);

SharedTimer.java
this.timerCleanup = LazyCleaner.getInstance().register(refCount, new TimerCleanup(timer));

    So it's:

    • Every PgConnection registers itself with a PgConnectionCleaningAction
    • Every psql StreamWrapper registers itself with a TempFileHolder (implements CleaningAction) to close streams and delete temp files
    • Every psql SharedTimer registers itself with a TimerCleanup to cancel the timer

    The latter two are come-and-go registrations, but the PgConnection is an issue. In a setup with

    • Tomcat with postgresql-42.6.0.jar in /lib for the common classloader
    • Multiple webapps using PSQL
    • Each webapp using a connection pool which keeps connections alive for a while

    each webapp will have PgConnection instances registered in the LazyCleaner, which is shared as common lib (by design). Thus re-deploying one webapp will not stop the LazyCleaner thread, as the other webapp(s) still have PgConnections registered.

    A Thread will inherit an AccessControlContext through its stacktrace on start, the effect of which is that the initial LazyCleaner thread will hold a reference to the ProtectionDomain of the webapp that initially started the thread. As described above, the LazyCleaner thread then never exits, and thus keeps that webapp classloader from gc.

    For now, a downgrade to postgresql-42.5.6.jar solves the problem.