Kryo serialization caused fatal java runtime error with custom objects

Question

I am trying to serialize hashmaps and collections of two custom classes (containing more hashmaps and collections).

Class1: NodeStorage.java

@NotNull
private final String id;

@Nullable
private String type;

@Nullable
private HashMap<String, String> properties;

Class2: RelationshipStorage.java

    @NotNull
private final String id;

@Nullable
private String type;

@Nullable
private HashMap<String, String> properties;

@NotNull
private final NodeStorage startNode;

@NotNull
private final NodeStorage endNode;

The collections to serialize:

private HashMap<NodeStorage, NodeStorage> readsSetNode;
private HashMap<NodeStorage, NodeStorage> updateSetNode;
private ArrayList<NodeStorage>            deleteSetNode;
private ArrayList<NodeStorage>            createSetNode;

private HashMap<RelationshipStorage, RelationshipStorage> readsSetRelationship;
private HashMap<RelationshipStorage, RelationshipStorage> updateSetRelationship;
private ArrayList<RelationshipStorage>                    deleteSetRelationship;
private ArrayList<RelationshipStorage>                    createSetRelationship;

What I tried until now:

 kryo.register(NodeStorage.class, 1);
    kryo.register(RelationshipStorage.class, 2);
    kryo.register(HashMap.class, mapSerializer);

    mapSerializer.setKeyClass(NodeStorage.class, kryo.getSerializer(NodeStorage.class));
    mapSerializer.setKeyClass(RelationshipStorage.class, kryo.getSerializer(RelationshipStorage.class));
    mapSerializer.setValuesCanBeNull(false);
    mapSerializer.setKeysCanBeNull(false);
    listSerializer.setElementClass(NodeStorage.class, kryo.getSerializer(NodeStorage.class));
    listSerializer.setElementClass(RelationshipStorage.class, kryo.getSerializer(RelationshipStorage.class));
    listSerializer.setElementsCanBeNull(false);


    public byte[] serialize()
{
    ByteArrayOutputStream stream = new ByteArrayOutputStream();
    Output output = new Output(stream);
    mapSerializer.write(kryo, output, readsSetNode);
    byte[] bytes = output.toBytes();
    output.close();
    return bytes;
}

I tried it with kryo.writeclassandobject but it didn't work as well. I get:

> > #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00007f92f7f6efe0, pid=4637, tid=0x00007f92f94fd700
> #
> # JRE version: OpenJDK Runtime Environment (8.0_102-b14) (build 1.8.0_102-b14)
> # Java VM: OpenJDK 64-Bit Server VM (25.102-b14 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # V  [libjvm.so+0x787fe0]
> #
> # Core dump written

Full code at: https://github.com/Raycoms/thesis

Declaration:

private Kryo kryo = new Kryo();
MapSerializer        mapSerializer  = new MapSerializer();
CollectionSerializer listSerializer = new CollectionSerializer();

Answer 1

Kryo is not thread-safe, and practically any Java program has multiple threads going. For example, your log file shows that you have 25 threads running at the time of the crash. Even though your Kryo instance is private, the error is down deep in the bowels of Kryo, and there may be thread or JVM interactions there that are not under your control.

Try the thread-pool approach, quoted below, detailed in the Kryo readme:

import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.pool.*;

KryoFactory factory = new KryoFactory() {
  public Kryo create () {
    Kryo kryo = new Kryo();
    // configure kryo instance, customize settings
    return kryo;
  }
};
// Build pool with SoftReferences enabled (optional)
KryoPool pool = new KryoPool.Builder(factory).softReferences().build();

Then, in your serialize() function:

Kryo kryo = pool.borrow();
<... the rest of your code in serialize() before the return...>
pool.release(kryo);
return bytes;

As the README points out, you can alternatively use a callback:

// or use a callback to work with kryo - no need to borrow/release,
// that's done by `run`.
String value = pool.run(new KryoCallback() {
  public String execute(Kryo kryo) {
    return kryo.readObject(input, String.class);
  }
});

OP's final code

Posted here from the OP's comment so that others can read it more easily. The OP was not able to use mapSerializer in the end, but was able to serialize using this code:

private byte[] serialize() { 
    KryoPool pool = new KryoPool.Builder(factory).softReferences().build();
    Kryo kryo = pool.borrow();
    Output output = new Output(0, 1024);
    kryo.writeClassAndObject(output, readsSetNode);
    byte[] bytes = output.toBytes();
    output.close();
    pool.release(kryo);
    return bytes;
}

Note As @MartinGrotzke pointed out, per this issue, if you use kryo.register(class, id), make sure id>=10.