How do I fix the following serialization error when using a DataStreamer with a StreamReceiver? I am guessing that it is not able to find the class when deserializing the RowStreamReceiver.
Error: SEVERE: Failure in Java callback class org.apache.ignite.IgniteException: Platform error:Apache.Ignite.Core.Binary.BinaryObjectException: Unknown pair [platformId=1, typeId=113114]
I'm using Apache Ignite 2.0, and I am trying to employ the same kind of code demonstrated in this test: https://github.com/apache/ignite/blob/master/modules/platforms/dotnet/Apache.Ignite.Core.Tests/Dataload/DataStreamerTest.cs#L436
I've tried adding the assembly to the configuration, but that didn't seem to help. config.Assemblies.Add(typeof(RowStreamReceiver).Assembly.FullName);
Here's the relevant code:
DataStreamer:
using (var ds = m_ignite.GetDataStreamer<string, IBinaryObject>(CacheName)) {
ds.AllowOverwrite = true;
ds.Receiver = new RowStreamReceiver(); // If I comment this out, the error goes away
Parallel.ForEach(rows.Select((r, i) => new KeyValuePair<long, string>(i, r)), r => {
var pair = BuildRow(r.Key, r.Value);
ds.AddData(pair);
});
}
StreamReceiver:
[Serializable]
public class RowStreamReceiver : IStreamReceiver<string, IBinaryObject> {
public void Receive(ICache<string, IBinaryObject> cache, ICollection<ICacheEntry<string, IBinaryObject>> entries) {
var bin = cache.Ignite.GetBinary();
cache.PutAll(entries.ToDictionary(x => x.Key, x => {
var builder = bin.GetBuilder(x.Value);
SetColumnFields(builder);
return builder.Build();
}));
}
private static void SetColumnFields(IBinaryObjectBuilder builder) {
/* logic to set fields */
}
}
Stack Trace:
Jul 07, 2017 11:25:22 AM java.util.logging.LogManager$RootLogger log
SEVERE: Failure in Java callback class org.apache.ignite.IgniteException: Platform error:Apache.Ignite.Core.Binary.BinaryObjectException: Unknown pair [platformId=1, typeId=113114] ---> Apache.Ignite.Core.Common.JavaException: class org.apache.ignite.binary.BinaryObjectException: Unknown pair [platformId=1, typeId=113114]
at org.apache.ignite.internal.processors.platform.binary.PlatformBinaryProcessor.processInStreamOutStream(PlatformBinaryProcessor.java:119)
at org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutStream(PlatformTargetProxyImpl.java:155)
at org.apache.ignite.internal.processors.platform.callback.PlatformCallbackUtils.inLongLongLongObjectOutLong(Native Method)
at org.apache.ignite.internal.processors.platform.callback.PlatformCallbackGateway.dataStreamerStreamReceiverInvoke(PlatformCallbackGateway.java:464)
at org.apache.ignite.internal.processors.platform.datastreamer.PlatformStreamReceiverImpl.receive(PlatformStreamReceiverImpl.java:100)
at org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:137)
at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.localUpdate(DataStreamProcessor.java:382)
at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.processRequest(DataStreamProcessor.java:301)
at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.access$000(DataStreamProcessor.java:58)
at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$1.onMessage(DataStreamProcessor.java:88)
at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1257)
at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:885)
at org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:114)
at org.apache.ignite.internal.managers.communication.GridIoManager$7.run(GridIoManager.java:802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: Unknown pair [platformId=1, typeId=113114]
at org.apache.ignite.internal.MarshallerContextImpl.getClassName(MarshallerContextImpl.java:385)
at org.apache.ignite.internal.processors.platform.binary.PlatformBinaryProcessor.processInStreamOutStream(PlatformBinaryProcessor.java:113)
... 16 more
--- End of inner exception stack trace --- at Apache.Ignite.Core.Impl.Unmanaged.UnmanagedCallbacks.Error(Void* target, Int32 errType, SByte* errClsChars, Int32 errClsCharsLen, SByte* errMsgChars, Int32 errMsgCharsLen, SByte* stackTraceChars, Int32 stackTraceCharsLen, Void* errData, Int32 errDataLen) at Apache.Ignite.Core.Impl.Unmanaged.IgniteJniNativeMethods.TargetInStreamOutStream(Void* ctx, Void* target, Int32 opType, Int64 inMemPtr, Int64 outMemPtr) at Apache.Ignite.Core.Impl.PlatformTarget.DoOutInOp[TR](Int32 type, Action`1 outAction, Func`2 inAction) at Apache.Ignite.Core.Impl.Binary.BinaryProcessor.GetType(Int32 id) at Apache.Ignite.Core.Impl.Binary.Marshaller.GetDescriptor(Boolean userType, Int32 typeId, Boolean requiresType) at Apache.Ignite.Core.Impl.Binary.BinaryReader.ReadFullObject[T](Int32 pos) at Apache.Ignite.Core.Impl.Binary.BinaryReader.TryDeserialize[T](T& res) at Apache.Ignite.Core.Impl.Binary.BinaryReader.Deserialize[T]() at Apache.Ignite.Core.Impl.Binary.BinaryReader.ReadBinaryObject[T](Boolean do Detach) at Apache.Ignite.Core.Impl.Binary.BinaryReader.TryDeserialize[T](T& res) at Apache.Ignite.Core.Impl.Binary.BinaryReader.Deserialize[T]() at Apache.Ignite.Core.Impl.Datastream.StreamReceiverHolder.InvokeReceiver[TK, TV](IStreamReceiver`2 receiver, Ignite grid, IUnmanagedTarget cache, IBinaryStream stream, Boolean keepBinary) at Apache.Ignite.Core.Impl.Datastream.StreamReceiverHolder.Receive(Ignite grid, IUnmanagedTarget cache, IBinaryStream stream, Boolean keepBinary) at Apache.Ignite.Core.Impl.Unmanaged.UnmanagedCallbacks.DataStreamerStreamReceiverInvoke(Int64 memPtr, Int64 unused, Int64 unused1, Void* cache) at Apache.Ignite.Core.Impl.Unmanaged.UnmanagedCallbacks.InLongLongLongObjectOutLong(Void* target, Int32 type, Int64 val1, Int64 val2, Int64 val3, Void* arg)
at org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:497)
at org.apache.ignite.internal.processors.platform.callback.PlatformCallbackUtils.inLongLongLongObjectOutLong(Native Method)
at org.apache.ignite.internal.processors.platform.callback.PlatformCallbackGateway.dataStreamerStreamReceiverInvoke(PlatformCallbackGateway.java:464)
at org.apache.ignite.internal.processors.platform.datastreamer.PlatformStreamReceiverImpl.receive(PlatformStreamReceiverImpl.java:100)
at org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:137)
at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.localUpdate(DataStreamProcessor.java:382)
at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.processRequest(DataStreamProcessor.java:301)
at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.access$000(DataStreamProcessor.java:58)
at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$1.onMessage(DataStreamProcessor.java:88)
at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1257)
at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:885)
at org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:114)
at org.apache.ignite.internal.managers.communication.GridIoManager$7.run(GridIoManager.java:802)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
The "Unknown pair" issue is caused by not using GetDataStreamer correctly. My cache was originally created like this:
ignite.GetOrCreateCache<string, object>(cacheConfig)
so when getting the DataStreamer, I needed to use the same types, and then use KeepWithBinary
var ds = m_ignite.GetDataStreamer<string, object>(CacheName).KeepWithBinary<string, IBinaryObject>()