What is the best way to serialize class hierarchies in Hazelcast?

Question

I would like to serialize and cache in Hazelcast an @Entity that has a bidirectional relation with another @Entity.

I am looking for what is considered offical best practise in such a scenario - a solution that would:

1. correctly serialize a class hierarchy that contains possibly null references as well as references to sets and lists
2. handle reference cycles in the graph

So far I succeeded in the former by implementing DataSerializable in every @Entity in my graph. I handle nulls and sets in the following way:

@Override
public void writeData(ObjectDataOutput out) throws IOException {
    out.writeBoolean(certificationNumber != null);
    if (certificationNumber != null)
        out.writeUTF(certificationNumber);
    out.writeShort(getSkills().size());
    for (SkillEntity s: getSkills())
        s.writeData(out);
}


@Override
public void readData(ObjectDataInput in) throws IOException {
    if (in.readBoolean()) {
        certification = new CertificationEntity();
        certification.readData(in);
    }
    short size = in.readShort();
    skills = new HashSet<>();
    for(int i=0; i<size; i++)
        skills.add(in.readObject());
}

However breaking reference cycles is bit more challenging. This thread suggests:

I think it would be based on maintaining a threadlocal map.

On serialization you need to lookup of your object to serialize already is in that map.. if so.. you need to serialize some kind of placeholder (e..g a uuid)

If it isn't in that map, generate a placeholder and put it in that map and write the actual object and you probably want to write the placeholder so you can use this information when deserializing.

I think this should put you in the right direction, but since I have not implemented this before I don't know if there are any gotcha's here.

While the suggestion above makes perfect sense I am not quite sure how to go about doing it i.e. what software engineering pattern to apply and how to cleanly separate my entities from the serialization logic.

Any pointers?

Answer 1

I think the answer to my question is on page 201 of 'Mastering Hazelcast 3.9' handbook that can be downloaded from Hazelcast website.

The contents of the page:

public class PersonKryoSerializer implements StreamSerializer<Person> {
    private static final ThreadLocal<Kryo> kryoThreadLocal
            = new ThreadLocal<Kryo>() {
        @Override
        protected Kryo initialValue() {
            Kryo kryo = new Kryo();
            kryo.register(Person.class);
            return kryo;
        }
    };
    @Override
    public int getTypeId() {
        return 2;
    }
    @Override
    public void write(ObjectDataOutput objectDataOutput, Person product)
            throws IOException {
        Kryo kryo = kryoThreadLocal.get();
        Output output = new Output((OutputStream) objectDataOutput);
        kryo.writeObject(output, product);
        output.flush();
    }
    @Override
    public Person read(ObjectDataInput objectDataInput)
            throws IOException {
        InputStream in = (InputStream) objectDataInput;
        Input input = new Input(in);
        Kryo kryo = kryoThreadLocal.get();
        return kryo.readObject(input, Person.class);
    }
    @Override
    public void destroy() {
    }
}

The PersonKryoSerializer is relatively simple to implement. The nice thing is that Kryo takes care of cycle detection and produces much smaller serialized data than Java serialization. For one of our customers we managed to reduce the size of map entries from a 15 kilobyte average using Java Serialization, to a less than six kilobyte average. When we enabled Kryo compression, we managed to get it below three kilobytes.