Apache Apex - kryo ArrayList Exception

Question

I've implemented an operator to deserialize from avro byte[] to Object. After that I sent the object to ConsoleOutputOperator.

public final transient DefaultInputPort<byte[]> input = new DefaultInputPort<byte[]>() {
    @Override
    public void process(byte[] tuple) {
        SpecificDatumReader reader = new SpecificDatumReader<T>(User.getClassSchema());
        BinaryDecoder binaryDecoder = DecoderFactory.get().binaryDecoder(tuple, null);
        try {
            binaryDecoder.readInt();
            obj = (T) reader.read(null, binaryDecoder);
        } catch (IOException e) {
            errorCount++;
            LOG.error("Avro decoder error.");
        }

        if (obj != null) {
            output.emit(""+obj);
            recordCount++;
        }else if (errorPort.isConnected()) {
            errorPort.emit(obj);
            errorCount++;
        }
    }
};

But, I got this error.

2016-05-09 10:53:21,481 ERROR com.datatorrent.stram.engine.StreamingContainer: deploy request failed
com.esotericsoftware.kryo.KryoException: Class cannot be created (missing no-arg constructor): org.apache.avro.generic.GenericData$Array
Serialization trace:
user (com.log.avro.User)
obj (com.log.avro.AvroOperator)
at  com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInstantiatorOf(Kryo.java:1228)
at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049)
at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058)
at com.esotericsoftware.kryo.serializers.CollectionSerializer.create(CollectionSerializer.java:89)
at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:93)
at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.java:193)
at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java:137)
at com.datatorrent.stram.engine.StreamingContainer.deployNodes(StreamingContainer.java:894)
at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingContainer.java:843)
at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatResponse(StreamingContainer.java:801)
at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(StreamingContainer.java:694)
at com.datatorrent.stram.engine.StreamingContainer.main(StreamingContainer.java:312)

I think that it doesn't not support "ArrayList".

kryo.readObject(input, ArrayList.class);

How to fix this error?

Answer 1

Take a look at: http://avro.apache.org/docs/1.8.0/api/java/org/apache/avro/generic/GenericData.html

You'll notice that GenericData.Array does not have a default constructor (i.e. a constructor with no arguments) which Kryo requires in order to serialize/deserialize.

You can use the annotation: @DefaultSerializer(JavaSerializer.class) if the class is Java serializable as is done in: library/src/main/java/com/datatorrent/lib/codec/KryoJdkContainer.java in Apex Malhar.

Please look at: http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception for additional options.