Search code examples
javagoogle-app-enginememory-leaksjdovisualvm

Is this evidence of a GAE memory leak?

Depending on the setting of autocreateDatastoreTxns, I get a memory leak, with one instance of each of the classes below created for each query (a read). i.e. 100 queries creates 100 instances of each of the classes below (with the exception of the DatastoreServiceConfig, which gets 2 instances per query).

I've found this using the Java VisualVM profiler in the development environment. The reason I'm doing this is that in production, our instance heap sizes keep growing (usually getting too large after 10-20 thousand requests) finally causing slow responses and instance restarts. I don't know if this is the cause, but it's the first leak I've been able to identify so far.

// Leaks with datanucleus.appengine.autoCreateDatastoreTxns=false

org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManager
org.datanucleus.store.appengine.KeyRegistry
org.datanucleus.store.appengine.EmualtedXARResource
org.datanucleus.store.appengine.DatastoreConnectionFactoryImpl$DatastoreManagedConnection

// Leaks with datanucleus.appengine.autoCreateDatastoreTxns=true

com.google.appengine.api.datastore.DatastoreServiceConfig  // 2 instances per query
org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManager
com.google.appengine.api.datastore.AsyncDatastoreServiceImpl
com.google.appengine.api.datastore.DatastoreServiceImpl
org.datanucleus.store.appengine.jdo. DatastoreJDOTransation
com.google.appengine.api.datastore.DatastoreXARResource
com.google.appengine.api.datastore.DatastoreProperty
com.google.appengine.api.datastore.KeyRegistry
com.google.appengine.api.datastore.DatastoreConnectionFactoryImpl$DatastoreManagedConnection
com.google.appengine.api.datastore.TransactionStackImpl$ThreadLocalTransactionStack$StaticMember
com.google.appengine.api.datastore.TransactionStackImpl
org.datanucleus.store.appengine.RuntimeExceptionWrapperingDatastoreService

Here is my code:

PersistenceManager pm = PMF.get().getPersistenceManager();
try {
    account = pm.detachCopy(pm.getObjectById(Account.class, accountKey));
} catch (javax.jdo.JDOObjectNotFoundException ex) {
    account = null;
} finally {
    pm.close();
}

Any ideas/thoughts? Is this a real memory leak in Google AppEngine, or is it just fact of life for the development environment, or maybe my own error?

Solution

  • The development appserver is intended to simulate the semantics of the real appserver, so that you can develop with reasonable fidelity. That doesn't include memory behavior, particularly with regard to the datastore. The dev server tends to keep things in memory in ways that the real appserver doesn't. Profiling the development server can still be useful for sorting out leaks on the app side, but isn't going to give you much guidance about leaks that may be on the appserver side. We do watch out for those, though. And heaps do fragment over time.

    Some apps, by the nature of their usage and data access patterns, benefit from larger front-end instances. Those cost more, so you have to test and weight the benefit against the added cost.