Java microstream List Object not stored correctly

Question

I have the below object structure which extends ArrayList. When it is retrieved the List is null and so none of the values are stored within the microstream object graph. Not sure if this is either

1. a bug
2. unsupported feature
3. A CustomHandler must be implemented

The code for creating the FooTable Object and toString results in

FooTable(super=TableListImpl(super=[a, b, c], tableNo=1, datatableNo=2), baz=baz)

        FooTable foo = new FooTable(1,2);
        foo.setBaz("baz");
        foo.add("a");
        foo.add("b");
        foo.add("c");
        System.out.println(foo1.toString());

Store FooTable in MicroStream. Stop/Start the app/DB and retrieve FooTable and the List is null. Interestingly when inspection the object variable 'size=3'. It appears microstream cannot see the fact this object extends List and only persists the other values ignoring the List.

Any advise on how to resolve this without changing the object structure.

Note: Lombok is being used here for brevity.

import lombok.Getter;
import lombok.Setter;
import lombok.ToString;

@Getter
@Setter
@ToString(callSuper = true)
public class FooTable extends TableListImpl<String> {

    public FooTable(int tableNo, int datatableNo) {
        super(tableNo, datatableNo);
    }
    
    private static final long serialVersionUID = 1L;
    private String baz;
    
}

import java.util.ArrayList;

import lombok.Getter;
import lombok.RequiredArgsConstructor;
import lombok.ToString;

@RequiredArgsConstructor
@Getter
@ToString(callSuper = true)
public abstract class TableListImpl<E> extends ArrayList<E> implements TableList<E> {

    private static final long serialVersionUID = 1L;
    private final int tableNo;
    private final int datatableNo;
    
}

import java.util.List;

public interface TableList<E> extends List<E>, Table {

}

public interface Table {

    public int getTableNo();
    public int getDatatableNo();
}

Answer 1

The issue is not really a bug. The fact that no list elements are stored is caused by the java.util.ArrayList implementation that marks the underlaying object array as transient. Therefore, Microstream does not persist it by default. But a custom type handler can solve that. Here is an example how such a type-handler for the class FooTable may look like:

import one.microstream.X;
import one.microstream.persistence.binary.internal.AbstractBinaryHandlerCustomCollection;
import one.microstream.persistence.binary.types.Binary;
import one.microstream.persistence.types.PersistenceLoadHandler;
import one.microstream.persistence.types.PersistenceReferenceLoader;
import one.microstream.persistence.types.PersistenceStoreHandler;

public class FooTableTypeHandler extends AbstractBinaryHandlerCustomCollection<FooTable>
{
    //the fields to be stored are:
    //private final int tableNo from TableListImpl
    //private final int datatableNo from TableListImpl;
    //private String baz from FooTable
    //transient Object[] elementData from ArrayList
    
    //define the binary layout used for storing the class FooTable
    private static final long BINARY_OFFSET_TABLE_NO      = 0;
    private static final long BINARY_OFFSET_DATA_TABLE_NO = BINARY_OFFSET_TABLE_NO + Integer.BYTES;
    private static final long BINARY_OFFSET_BAZ           = BINARY_OFFSET_DATA_TABLE_NO + Integer.BYTES;
    private static final long BINARY_OFFSET_ELEMENTS      = BINARY_OFFSET_BAZ + Binary.referenceBinaryLength(1);
            
    protected FooTableTypeHandler()
    {
        super (FooTable.class,
            SimpleArrayFields(
                CustomField(int.class, "tableNo"),
                CustomField(int.class, "datatableNo"),
                CustomField(String.class, "baz")
                )
            );
    }
        
    @Override
    public void iterateLoadableReferences(final Binary data, final PersistenceReferenceLoader iterator)
    {
        //register all referenced items that need to be restored too
        data.iterateListElementReferences(BINARY_OFFSET_ELEMENTS, iterator);
        iterator.acceptObjectId(data.read_long(BINARY_OFFSET_BAZ));
    }
    
    @Override
    public void store(final Binary data, final FooTable instance, final long objectId, final PersistenceStoreHandler<Binary> handler)
    {
        //store items in the list
        data.storeIterableAsList(
            this.typeId()         ,
            objectId              ,
            BINARY_OFFSET_ELEMENTS,
            instance              ,
            instance.size()       ,
            handler
        );
        
        //store int values directly
        data.store_int(BINARY_OFFSET_TABLE_NO     , instance.getTableNo());
        data.store_int(BINARY_OFFSET_DATA_TABLE_NO, instance.getDatatableNo());
        
        //store a reference to the String field "baz" and handle the String itself, if needed
        data.store_long(BINARY_OFFSET_BAZ         , handler.apply(instance.getBaz()));
    }

    @Override
    public FooTable create(final Binary data, final PersistenceLoadHandler handler)
    {
        //read the int values
        //create empty instance
        return new FooTable(
            data.read_int(BINARY_OFFSET_TABLE_NO),
            data.read_int(BINARY_OFFSET_DATA_TABLE_NO));
    }
    
    private long getElementCount(final Binary data)
    {
        return data.getListElementCountReferences(BINARY_OFFSET_ELEMENTS);
    }
    
    @Override
    public void updateState(final Binary data, final FooTable instance, final PersistenceLoadHandler handler)
    {
        // instance must be cleared in case an existing one is updated
        instance.clear();
                
        //get all list elements
        data.collectObjectReferences(
            BINARY_OFFSET_ELEMENTS,
            X.checkArrayRange(this.getElementCount(data)),
            handler,
            e ->
                instance.add((String) e)
        );
        
        //get "baz"
        instance.setBaz((String) handler.lookupObject(data.read_long(BINARY_OFFSET_BAZ)));
    }

}