Search code examples
java.net-coreignite

Apache Ignite Binary Serialization between .NET Core and Java is not working


Hello folks I have run into issue where I use a group of Apache Ignite (2.8.1) server nodes in .NET core to create a data grid and run queries to the grid via an Apache ignite java client. I have not problem at all writing data in binary mode to the grid and ask queries via the think layer provided. I use DBeaver to run queries and everything is fine as expected. The issue rise while I am trying to query data from a java client which complains about a conflict in cache ": Conflicts during configuration merge for cache MY_CAHE". Find the error message below:

Caused by: class org.apache.ignite.spi.IgniteSpiException: Conflicts during configuration merge for cache 'DOTNET_BINARY_CACHE' : 
TRADE conflict: 
keyType is different: local=Apache.Ignite.Core.Cache.Affinity.AffinityKey, received=org.apache.ignite.cache.affinity.AffinityKey
valType is different: local=Servicing.Agent4.Service.Implementation.Misc.Ignite.Trade, received=Servicing.Agent4.Core.Java.Models.Trade

Find my implemnetation in .NET and Java below:

  public static class IgniteUtils
    {
        const string CACHE_NAME = "DOTNET_BINARY_CACHE";

        public static IgniteConfiguration DefaultIgniteConfig()
        {
            return new IgniteConfiguration
            {
                BinaryConfiguration = new BinaryConfiguration
                {
                    NameMapper = new BinaryBasicNameMapper { IsSimpleName = true },
                    CompactFooter = true,
                    TypeConfigurations = new[] {
                        new BinaryTypeConfiguration(typeof(Trade)) {
                            Serializer = new IgniteTradeSerializer()
                        }
                    }
                },
                // omit jvm and network options
                IncludedEventTypes = EventType.All,
                Logger = new IgniteNLogLogger(),
                CacheConfiguration = new[]{

                    new CacheConfiguration{
                            Name = CACHE_NAME,
                            CacheMode = CacheMode.Partitioned,
                            Backups = 0,
                            QueryEntities = new[] { new QueryEntity(typeof(AffinityKey), typeof(Trade))}
                }
            }
            };
        }
    }

The setup of Apache Ignite is happen on class:

 public class IgniteService
    {
        public void Start()
        {
           IIgnite _ignite = Ignition.Start(IgniteUtils.DefaultIgniteConfig());        
        
            // Create new cache and configure queries for Trade binary types.
            // Note that there are no such classes defined.
            var cache0 = _ignite.GetOrCreateCache<AffinityKey, Trade>("DOTNET_BINARY_CACHE");

            // Switch to binary mode to work with data in serialized form.
             var cache = cache0.WithKeepBinary<AffinityKey, IBinaryObject>();

            // Clean up caches on all nodes before run.
            cache.Clear();

            // Populate cache with sample data entries.
               IBinary binary = cache.Ignite.GetBinary();

            cache[new AffinityKey(1, 1)] = binary.GetBuilder("TRADE")
                .SetField("Symbol", "James Wilson")
                .SetField("Id", 1)
                .SetField("Login", 123)
                .SetField("SourceId", 1)
                .Build();     
        }

Domain class below:

    public class Trade
    {
        [QuerySqlField(IsIndexed = true)]
        public int Id { set; get; }
        [QueryTextField]
        public string Symbol { set; get; }
        [QuerySqlField]
        public int Login { set; get; }
        [QuerySqlField(IsIndexed = true)]
        public int SourceId { get; set; }

        //omit constructor    
    }

The Java client code

public class IgniteScheduler {

    final String CACHE_NAME = "DOTNET_BINARY_CACHE";

    @PostConstruct
    public void start() {
        IgniteConfiguration cfg = new IgniteConfiguration();
        // Enable client mode.
        cfg.setClientMode(true);
        CacheConfiguration<AffinityKey<Integer>, Trade> cacheCfg = new CacheConfiguration<>();
        cacheCfg.setName(CACHE_NAME);
        cacheCfg.setCacheMode(CacheMode.PARTITIONED);
        cacheCfg.setBackups(0);
        cacheCfg.setQueryEntities(Arrays.asList(new QueryEntity(AffinityKey.class, Trade.class)));
        // Setting up an IP Finder to ensure the client can locate the servers.
        TcpDiscoveryMulticastIpFinder ipFinder = new TcpDiscoveryMulticastIpFinder();
        ipFinder.setAddresses(Collections.singletonList("127.0.0.1:47500..47509"));
        cfg.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(ipFinder));
        cfg.setCacheConfiguration(cacheCfg);
        // Configure Ignite to connect with .NET nodes
        cfg.setBinaryConfiguration(new BinaryConfiguration()
                .setNameMapper(new BinaryBasicNameMapper(true))
                 .setCompactFooter(true)
               BinaryTypeConfiguration(Trade.class.getSimpleName())))             
        );

        // Start Ignite in client mode.
        Ignite ignite = Ignition.start(cfg);
        
      // omit functional code
}

Domain class below:

@Data
public class Trade implements Serializable {

    @QuerySqlField(index = true)
    public int Id;
    @QueryTextField
    public String Symbol;
    @QuerySqlField
    public int Login;
    //@AffinityKeyMapped does not work as well
    @QuerySqlField(index = true)
    public int SourceId;

    // omit constructor
}

Debugging Info

  • OS: Windows 10 10.0 amd64
  • VM information: Java(TM) SE Runtime Environment 11.0.5+10-LTS Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS
  • Apache Ignite 2.8.1 version

Solution

    1. You don't need to provide cache configuration on all nodes. Cache configuration should be provided once on the node that starts the cache.

    Remove setCacheConfiguration on Java side and simply call ignite.cache(CACHE_NAME) there.

    1. The exception is caused by the fact that NameMapper does not apply to query entities, KeyType and ValueType always use full type name (with namespace).

    Normally this is not an issue, and you can omit namespace in queries: select name from Trade works in your example (given that Trade has Name property).

    Working example: https://gist.github.com/ptupitsyn/a2c13f47e19ccfc9c0b548cf4d4fa629