How to use BinaryHandler correctly in Microstream to store volatile array of longs?

Question

I tried to store Eclipse Collection's LongArrayList (https://www.eclipse.org/collections/) with Microstream, but due to it's "items" field being transient this didn't work out of the box.

So what I did was to create a new class PersistableLongArrayList like this:

public class PersistableLongArrayList extends LongArrayList {
    static BinaryTypeHandler<PersistableLongArrayList> provideTypeHandler() {
        return Binary.TypeHandler(PersistableLongArrayList.class,
            Binary.Field_int(
                "size", 
                list -> list.size, 
                (list, value) -> list.size = value
            ),
            Binary.Field(
                long[].class, "items", 
                list -> list.items, 
                (list, value) -> list.items = value
            )
        );
    } 
}

The items field is now not null when loaded from a stored instance, however if I change the value afterwards and call storage.store(list), shutdown the db and restart it the new value is not stored, only size is stored correctly.

I've added a very simple example that shows this behavior:

public class SimpleTest {
    private PersistableLongArrayList root;

    public static void main(String[] args) {
        try {
            SimpleTest t = new SimpleTest();
            // DB erstellen
            EmbeddedStorageManager storage = t.startDB();
            // modify the list
            t.root.add(t.root.size() + 1);
            storage.store(t.root);
            // stop DB
            t.stopDB(storage);
            // restart DB
            storage = t.startDB();
            // show root element
            System.out.println(t.root);
            // Observed behavior: list "grows" from s.th. like this: [1] to s.th. like this [1, 0, 0, 0] after several runs (Probably only size grows and is stored correctly and therefore the fields up to "size" are printed but empty)
            // Expected beahvior: list starts with [1] and grows to something like this [1, 2, 3, 4] after several runs
        } catch (Throwable t) {
            t.printStackTrace();
        } finally {
            System.exit(0);
        }
    }

    private EmbeddedStorageManager startDB() {
        EmbeddedStorageManager storageManager = EmbeddedStorage.start();
        if(storageManager.root() == null) {
            PersistableLongArrayList list = new PersistableLongArrayList();
            storageManager.setRoot(list);
            storageManager.storeRoot();
            root = list;
        } else {
            root = (PersistableLongArrayList)storageManager.root();
        }
        return storageManager;
    }

    private void stopDB(EmbeddedStorageManager storageManager) {
        storageManager.shutdown();
    }
}

Probably I just misunderstood how to use the custom BinaryHandler correctly, but I have no idea what to change right now. Any advice is appreciated :)

Kind regards, Thomas

Answer 1

Using a custom PersistenceEagerStoringFieldEvaluator and PersistenceFieldEvaluator it is possible to persist the LongArrayList directly:

public class PersistenceFieldEvaluatorCustom implements PersistenceFieldEvaluator {

    @Override
    public boolean applies(final Class<?> entityType, final Field field) {

        //return true if field should be persisted, false if not
        if(entityType == org.eclipse.collections.impl.list.mutable.primitive.LongArrayList.class && field.getName().equals("items")) {
            return true;
        }

        //default: return false if field has the transient modifier
        return !XReflect.isTransient(field);
    }
}

public class PersistenceEagerStoringFieldEvaluatorCustom implements PersistenceEagerStoringFieldEvaluator {

    @Override
    public boolean isEagerStoring(final Class<?> t, final Field u) {

        //return true if field should be persisted at every store
        if(t == org.eclipse.collections.impl.list.mutable.primitive.LongArrayList.class && u.getName().equals("items")) {
            return true;
        }
        return false;
    }
}

Setup:

final EmbeddedStorageManager s = EmbeddedStorage.Foundation()
                .onConnectionFoundation(
                        c->{ c.setFieldEvaluatorPersistable(new PersistenceFieldEvaluatorCustom());
                             c.setReferenceFieldEagerEvaluator(new PersistenceEagerStoringFieldEvaluatorCustom());
                        })
                .start();

best regards

see https://github.com/microstream-one/microstream/discussions/247 for some more details