Search code examples
c++.netignite

Apache Ignite C++ complex type retrieved by .NET client fails with "Requesting mapping from grid failed" exception

Apache Ignite 2.14.0 server with .NET and C++ thick client "client" nodes...

Following the notes on platform interoperability and serialization on the Apache Ignite docs site, I have several services running (the Apache Ignite server running under Java, a .NET Core 6.0 ASP.NET Core web api and C++ and C# "worker" programs). I have defined a WorkerStatus class in both C# and C++ following the notes. I can push WorkerStatus into the cache from both C++ and C#, but I can only retrieve C# worker statuses from the web api service... if I run the C++ worker instead and try to retrieve from the cache, I get this exception:

Apache.Ignite.Core.Binary.BinaryObjectException: Requesting mapping from grid failed for [platformId=1, typeId=959909104]

I have the following for the binary configuration in the Ignite config file:

    <property name="binaryConfiguration">
        <bean class="org.apache.ignite.configuration.BinaryConfiguration">
            <property name="compactFooter" value="false"/>

            <property name="idMapper">
                <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                    <property name="lowerCase" value="true"/>
                </bean>
            </property>
                
            <property name="nameMapper">
                <bean class="org.apache.ignite.binary.BinaryBasicNameMapper">
                    <property name="simpleName" value="true"/>
                </bean>
            </property>
        </bean>
    </property>

I have tried simplifying the WorkerStatus classes, moving them in and out of packages or removing properties (like timestamps) that might be problematic. My suspicion is that for some reason (developer error?) the TypeId is not being calculated the same between C# and C++... that will be my next path of investigation, but my main interest is whether anyone has an existence proof that .NET Core (6.0) and C++ will play together...

Code is available at https://github.com/osmedd/ignite-demo.

The full exception:

PS D:\source\repos\ignite-demo> curl http://localhost:5018/configurator/workers
Apache.Ignite.Core.Binary.BinaryObjectException: Requesting mapping from grid failed for [platformId=1, typeId=959909104]
 ---> Apache.Ignite.Core.Common.JavaException: class org.apache.ignite.binary.BinaryObjectException: Requesting mapping from grid failed for [platformId=1, typeId=959909104]
        at org.apache.ignite.internal.processors.platform.binary.PlatformBinaryProcessor.processInStreamOutStream(PlatformBinaryProcessor.java:146)
        at org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutStream(PlatformTargetProxyImpl.java:136)
Caused by: java.lang.ClassNotFoundException: Requesting mapping from grid failed for [platformId=1, typeId=959909104]
        at org.apache.ignite.internal.MarshallerContextImpl.getClassName(MarshallerContextImpl.java:437)
        at org.apache.ignite.internal.MarshallerContextImpl.getClassName(MarshallerContextImpl.java:392)
        at org.apache.ignite.internal.processors.platform.binary.PlatformBinaryProcessor.processInStreamOutStream(PlatformBinaryProcessor.java:141)
        ... 1 more

   at Apache.Ignite.Core.Impl.Unmanaged.Jni.Env.ExceptionCheck()
   at Apache.Ignite.Core.Impl.Unmanaged.Jni.Env.CallVoidMethod(GlobalRef obj, IntPtr methodId, Int64* argsPtr)
   at Apache.Ignite.Core.Impl.Unmanaged.UnmanagedUtils.TargetInStreamOutStream(GlobalRef target, Int32 opType, Int64 inMemPtr, Int64 outMemPtr)
   at Apache.Ignite.Core.Impl.PlatformJniTarget.InStreamOutStream[TR](Int32 type, Action`1 writeAction, Func`2 readAction, Func`2 errorAction)
   --- End of inner exception stack trace ---
   at Apache.Ignite.Core.Impl.PlatformJniTarget.InStreamOutStream[TR](Int32 type, Action`1 writeAction, Func`2 readAction, Func`2 errorAction)
   at Apache.Ignite.Core.Impl.PlatformTargetAdapter.DoOutInOp[TR](Int32 type, Action`1 outAction, Func`2 inAction, Func`2 errorAction)
   at Apache.Ignite.Core.Impl.Binary.BinaryProcessor.GetTypeName(Int32 id, Byte platformId, Func`2 errorAction)
   at Apache.Ignite.Core.Impl.Binary.Marshaller.GetTypeName(Int32 typeId)
   at Apache.Ignite.Core.Impl.Binary.Marshaller.GetDescriptor(Boolean userType, Int32 typeId, Boolean requiresType, String typeName, Type knownType)
   at Apache.Ignite.Core.Impl.Binary.BinaryReader.ReadFullObject[T](Int32 pos, Type typeOverride)
   at Apache.Ignite.Core.Impl.Binary.BinaryReader.TryDeserialize[T](T& res, Type typeOverride)
   at Apache.Ignite.Core.Impl.Binary.BinaryReader.Deserialize[T](Type typeOverride)
   at Apache.Ignite.Core.Impl.Binary.BinaryReader.ReadBinaryObject[T](Boolean doDetach)
   at Apache.Ignite.Core.Impl.Binary.BinaryReader.TryDeserialize[T](T& res, Type typeOverride)
   at Apache.Ignite.Core.Impl.Binary.BinaryReader.Deserialize[T](Type typeOverride)
   at Apache.Ignite.Core.Impl.Binary.BinaryReader.ReadObject[T]()
   at Apache.Ignite.Core.Impl.Cache.CacheEnumerator`2.<MoveNext>b__4_0(IBinaryStream stream)
   at Apache.Ignite.Core.Impl.PlatformJniTarget.OutStream[T](Int32 type, Func`2 readAction)
   at Apache.Ignite.Core.Impl.PlatformTargetAdapter.DoInOp[T](Int32 type, Func`2 action)
   at Apache.Ignite.Core.Impl.Cache.CacheEnumerator`2.MoveNext()
   at Apache.Ignite.Core.Impl.Cache.CacheEnumeratorProxy`2.MoveNext()
   at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
   at System.Text.Json.Serialization.Converters.IEnumerableDefaultConverter`2.OnWriteResume(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryWrite(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.WriteCore(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.WriteCoreAsObject(Utf8JsonWriter writer, Object value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.JsonSerializer.WriteCore[TValue](JsonConverter jsonConverter, Utf8JsonWriter writer, TValue& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.JsonSerializer.WriteStreamAsync[TValue](Stream utf8Json, TValue value, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken)
   at System.Text.Json.JsonSerializer.WriteStreamAsync[TValue](Stream utf8Json, TValue value, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken)
   at System.Text.Json.JsonSerializer.WriteStreamAsync[TValue](Stream utf8Json, TValue value, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResultFilterAsync>g__Awaited|30_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Suggestions? Pointers to more complex examples than are shown on the Ignite site? Does moving to an SQL approach avoid this serialization code at the cost of the convenience of nice objects to work with?

Thanks for any thoughts...

Solution

  • The fix is to add model types to BinaryConfiguration, and ensure matching IdMapper and NameMapper (those must be set both in Spring XML and in .NET code):

    var cfg = new IgniteConfiguration
{
    BinaryConfiguration = new BinaryConfiguration(typeof(Organization))
    {
        CompactFooter = false,
        NameMapper = new BinaryBasicNameMapper { IsSimpleName = true }
    },
    SpringConfigUrl = "config.xml"
};

    Where config.xml is:

    <?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util
        http://www.springframework.org/schema/util/spring-util.xsd">
    <bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
        <property name="binaryConfiguration">
            <bean class="org.apache.ignite.configuration.BinaryConfiguration">
                <property name="compactFooter" value="false"/>
                <property name="idMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicIdMapper">
                        <property name="lowerCase" value="true"/>
                    </bean>
                </property>

                <property name="nameMapper">
                    <bean class="org.apache.ignite.binary.BinaryBasicNameMapper">
                        <property name="simpleName" value="true"/>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>

    Explanation: normally, Ignite will register those types in the cluster automatically on the first use. When only one language is involved (e.g. only C# or only Java), it works seamlessly.

    However, when platform/language interop is inviolved, explicit registration is often required. For example, if you put data into Ignite from Java, it will register the Java type in the cluster. But if you try to read it from .NET, it does not yet know about any .NET types, so this error may occur.