Why are Collections retrieved as null?

Question

I have a Google App Engine project using JDO and when I retrieve a class containing collections of collections, each collection is null while simpler fields in the class are retrieved OK.

I'm working on the basis I'm using the wrong annotations and so the enhancer is politely ignoring the fields at run-time - but can't see where.

When I debug through, I can see my call to makePersistent updates the references to these fields and they remain non-null and populated. When I use the ID from the persisted object to retrieve it again then the fields are null.

My persistent object sub-classes a generic persistent object, below.

@PersistenceCapable
@Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE)
public abstract class PersistentObject {
    @PrimaryKey
    @Persistent(valueStrategy=IdGeneratorStrategy.IDENTITY)
    protected Long id;

    // getters and setters omitted
}

The sub-class I'm trying to store and retrieve looks like this. The name field (case 1) is retrieved OK, but taskToCommandMap, taskToDefaultCommandParams and taskToTransitions all come back as null.

I've tried marking the collections of collections as serializable="true" because of the DataNucleus comment on this question.

@PersistenceCapable
public class ConcretePersistentObject extends PersistentObject {
    /** Case 1 - Stored and retrieved fine */
    @Persistent 
    private String name;

    /** Case 2 - A new object ref is visible in the debugger for this after the call to makePersistent , but when retrieved it's null */
    @Persistent 
    private Map<String, String> taskToCommandMap;

    /** Case 3 -  DataSelection is serializable */
    @Persistent 
    private Map<String, Map<String, DataSelection>> taskToDefaultCommandParams;

    /** Case 4 - Serialized because GAE doesn't support persisting Map values of type LinkedList or ArrayList. 
     */
    @Persistent(serialized="true")
    private Map<String, List<String>> taskToTransitions;

    // getters and setters omitted

DataSelection referenced in there is an interface that extends Serializable. None of the classes that implement DataSelection are referenced in the annotations and none of them are @PersistenceCapable.

public interface DataSelection extends Serializable {...}

Edit 1: The actual retrieval happens in a generic DAO,

public T get(Long id) {
    if(id == null) {
        return null;
    }
    PersistenceManager pm = pmf.getPersistenceManager();
    T persistentObject = null;
    try {
        persistentObject = (T) pm.getObjectById(persistentClass, id);

    } catch (Exception e) {
        logger.warning( e.toString() );
        e.printStackTrace();
        return null;

    } finally {
        pm.close();
    }

    return persistentObject;
}

I build and store an instance, then call the get method and then test the fields one by one:

ConcretePersistentObject cpo1 = buildAndStore();
ConcretePersistentObject cpo2 = concretePersistentObjectDao.get( cpo1.getId() );
assertEquals(cpo1.getName(), cpo2.getName(); // passes
assertEquals(true, cpo1.getTaskToCommandMap().keySet().equals(cpo2.getTaskToCommandMap().keySet())); // null pointer exception

At this point, cpo1 is populated with the original data. It has a reference to a HashMap with data in it. cpo2 is an object, it has the simple fields set but the Maps are now nulls.

Why am I seeing nulls returned for these fields? What should I be doing differently? ...and where's the rum?

Edit 2: Both objects are in a TRANSIENT state at the time of comparison.

I don’t set detachAllOnClose anywhere, System.out.println( pmfInstance.getProperties() ) doesn’t show it set, and the DataNucleus doco shows that it defaults to false.

Right before calling pm.getObjectById the FetchPlan looks like this:

FetchPlan.DetachmentOptions = 1
FetchPlan.MaxFetchDepth = 1
FetchPlan.FetchSize = 0

datanucleus.retainValues = true in the PMF properties

Answer 1

Thanks to Neil Stockton's comments, the unit test now passes.

Fetch Plans are documented here: http://www.datanucleus.org/products/datanucleus/jdo/fetchgroup.html

The null fields weren't loaded they're not the among the data types that are loaded by default (primitives, String, Date, Object wrappers of primitives, etc.)

To make the code above work, I moved each of the null fields into the default fetch group by specifying a new attribute on the @Persistance annotation - defaultFetchGroup="true"

@PersistenceCapable
public class ConcretePersistentObject extends PersistentObject {
    /** Case 1 - Stored and retrieved fine */
    @Persistent 
    private String name;

    /** Case 2 - A new object ref is visible in the debugger for this after the call to makePersistent , but when retrieved it's null */
    @Persistent(defaultFetchGroup="true") 
    private Map<String, String> taskToCommandMap;

    /** Case 3 -  DataSelection is serializable */
    @Persistent(defaultFetchGroup="true")
    private Map<String, Map<String, DataSelection>> taskToDefaultCommandParams;

    /** Case 4 - Serialized because GAE doesn't support persisting Map values of type LinkedList or ArrayList. 
     */
    @Persistent(serialized="true", defaultFetchGroup="true")
    private Map<String, List<String>> taskToTransitions;

    // getters and setters omitted