Search code examples
scalaserializationkryo

Deserializing LongMap in Kryo

I have a class with a field of scala.collection.mutable.LongMap type.

After serializing it with Kryo I attempt to deserialize the object and get the following exception:

com.esotericsoftware.kryo.KryoException: java.lang.IllegalArgumentException: Can not set final scala.collection.mutable.LongMap field com.name.of.field to scala.collection.mutable.HashMap
Serialization trace:
field (com.name.of)
    at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.read(FieldSerializer.java:626) ~[com.esotericsoftware.kryo.kryo-2.21.jar:na]
    at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:221) ~[com.esotericsoftware.kryo.kryo-2.21.jar:na]
    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:648) ~[com.esotericsoftware.kryo.kryo-2.21.jar:na]
    at com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.read(FieldSerializer.java:605) ~[com.esotericsoftware.kryo.kryo-2.21.jar:na]
    at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:221) ~[com.esotericsoftware.kryo.kryo-2.21.jar:na]
    at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:729) ~[com.esotericsoftware.kryo.kryo-2.21.jar:na]

IIUC the LongMap is serialized as HashMap and then deserialization fails as the HashMap can't be written to the LongMap field.

I manually ran something like https://github.com/romix/akka-kryo-serialization/blob/master/src/test/scala/com/romix/scala/serialization/kryo/MapSerializerTest.scala#L78 and confirmed that serialized LongMap is deserialized into a HashMap.

Any idea how to properly read/write this Object so the LongMap will be read as LongMap instead of HashMap?

Do I need to use a proxy class? write a custom serializer/deserializer? Alternatively, is there a decent serialization library that handles LongMaps properly?

P.S. I would have tagged the question with LongMap but I don't have enough reputation to create new tags.

Solution

  • Yes, you need to add a custom serializer. There is https://github.com/twitter/chill#serializers-for-scala-classes which includes serializers for some types in Scala standard library, but apparently not for LongMap (you may already be using this library, perhaps indirectly). Look at how they do it and write your own.

    However, this error shouldn't happen by default. Look for Kryo#register and Kryo#setDefaultSerializer calls in your code (or code you call): are you telling Kryo to serialize/deserialize all scala.collection.mutable.Maps as HashMaps?