Search code examples
ignite

Communication between two Ignite clusters (maybe merging two Ignite clusters in one)

We have two Ignite clusters. They work completely independent one from other. This is how it works now:

IgniteCluster1

  • SpringBootApplication1 start up IgniteCluster1's server node
  • /request1 come to SpringBootApplication1
  • SpringBootApplication1 has InstanceOfIgniteCluster1
  • through InstanceOfIgniteCluster1 SpringBootApplication1 execute something like
String response1 = compute.execute(Task1.class, request1);
  • IgniteCluster1 has Cache1
  • method in Task1 class takes some data from Cache1 and do some calculations
  • response1 is send as response for /request1

IgniteCluster2 works the same way

  • SpringBootApplication2 start up its server node
  • /request2 come to SpringBootApplication2
  • SpringBootApplication2 has InstanceOfIgniteCluster2
  • through InstanceOfIgniteCluster2 SpringBootApplication2 execute something like
String response2 = compute.execute(Task2.class, request2);
  • IgniteCluster2 has Cache2
  • method in Task2 class takes some data from Cache2 and do some calculations
  • response2 is send as response for /request2

But now Task1 from IgniteCluster1 should have response2 for calculations of response1. We do not want to have Cache2 in IgniteCluster1 for that. So we must somehow get response2 from IgniteCluster2. Also we don't want send /request to SpringBootApplication2 across network because of latency - we want ask data from IgniteCluster2 directly.

We see now only one solution - all remain the same, but in SpringBootApplication1 we also start up client node of IgniteCluster2. And in Task1 we firstly get response2 through this client node

String response2 = compute.execute(Task2.class, request);

and after that we get response1 through server node of IgniteCluster1

String response1 = compute.execute(Task1.class, request + response2);
  1. Is it right solution?
  2. Is it the only solution?
  3. Is it the best solution?
  4. Actually for us it would be better if there would be only one IgniteCluster with nodes of 2 different type - some nodes started by SpringBootApplication1 (Type1) and some nodes started by SpringBootApplication2 (Type2). And nodes of different type could somehow communicate with each other (so Type1 node can ask data directly from Type2 nodes). And caches (Cache1 and Cache2) would be stored only on the appropriate nodes (so Cache1 on Type1 nodes and Cache2 on Type2 nodes). Is it possible? How?
Solution

  • 1.2.3. This solution will definitely work. But quality of this way depends on your service design/architecture.

    1. Yes, it's possible. You can colocate cache with a subset of cluster nodes. See this reference https://www.gridgain.com/docs/latest/developers-guide/configuring-caches/managing-data-distribution

    Optionally, after configure cache's node filter you can use ClusterGroup feature to compute task on nodes which contains nessesary cache's partitions. Like so:

    Ignite ignite = Ignition.ignite();
IgniteCluster cluster = ignite.cluster();

ClusterGroup cacheGroup1 = cluster.forCacheNodes("myCache1");
ClusterGroup cacheGroup2 = cluster.forCacheNodes("myCache2");

// Request to type1 nodes
String response1 = ignite.compute(cacheGroup1).execute(Task1.class, request1);

// type2
String response2 = ignite.compute(cacheGroup2).execute(Task2.class, request2);

    More details here https://www.gridgain.com/docs/latest/developers-guide/distributed-computing/cluster-groups