Search code examples
spring-bootignite

Apache Ignite replicated cache not available across clients


I have two clients and one server.

[14:02:16] Topology snapshot [ver=8, servers=1, clients=2, CPUs=8, heap=7.8GB]

I have created a replicated Cache that I would like both instances to have access to the same token cache, i.e tokens added by one client can be read by the other and vice versa.

private IgniteCache<Long, CacheToken> getOrCreateCache() {
    CacheConfiguration<Long, CacheToken> tokenCacheConfig = new CacheConfiguration<>(REVOCATION_CACHE);

    tokenCacheConfig.setCacheMode(CacheMode.REPLICATED);
    tokenCacheConfig.setIndexedTypes(Long.class, CacheToken.class);
    tokenCacheConfig.setExpiryPolicyFactory(CreatedExpiryPolicy.factoryOf(Duration.THIRTY_MINUTES));

    return ignite.getOrCreateCache(tokenCacheConfig);
}

What I'm actually seeing when I look at the cache is two separate caches with the same name are being created and not replicated.

Nodes for: TOKEN_REVOCATION_LIST(@c2)
+==============================================================================================+
|      Node ID8(@), IP       | CPUs | Heap Used | CPU Load |   Up Time    | Size | Hi/Mi/Rd/Wr |
+==============================================================================================+
| 0794F94D(@n2), xx.xx.xx.xx | 8    | 22.79 %   | 0.03 %   | 00:54:44:853 | 14   | Hi: 0       |
|                            |      |           |          |              |      | Mi: 0       |
|                            |      |           |          |              |      | Rd: 0       |
|                            |      |           |          |              |      | Wr: 0       |
+----------------------------+------+-----------+----------+--------------+------+-------------+
| 7B9F7E25(@n1), xx.xx.xx.xx | 8    | 23.12 %   | 0.13 %   | 00:55:00:366 | 0    | Hi: 0       |
|                            |      |           |          |              |      | Mi: 0       |
|                            |      |           |          |              |      | Rd: 0       |
|                            |      |           |          |              |      | Wr: 0       |
+----------------------------+------+-----------+----------+--------------+------+-------------+
| E7321C6D(@n0), xx.xx.xx.xx | 8    | 6.90 %    | 0.03 %   | 00:55:08:795 | 0    | Hi: 0       |
|                            |      |           |          |              |      | Mi: 0       |
|                            |      |           |          |              |      | Rd: 0       |
|                            |      |           |          |              |      | Wr: 0       |
+----------------------------------------------------------------------------------------------+

The behavior I expect to see is one shared cache, or at minimum, the tokens being replicated across the Nodes. From within each node, I can read the values that the node wrote, but not across nodes.

What changes to the configuration should I make to accomplish a shared, replicated cache? Are there any examples of this?

@Configuration
public class IgniteConfig {

@Bean
public Ignite ignite() {
    return Ignition.start(igniteConfiguration());
}

@Bean
public JdbcDataSource h2ExampleDb() {
    JdbcDataSource ds = new JdbcDataSource();
    ds.setURL("jdbc:h2:tcp://localhost/mem:ExampleDb");
    ds.setUser("sa");
    return ds;
}

@Bean
public IgniteConfiguration igniteConfiguration() {
    IgniteConfiguration ic = new IgniteConfiguration();
    ic.setClientMode(true);
    ic.setPeerClassLoadingEnabled(true);
    ic.setIncludeEventTypes(getEventTypes());
    ic.setDiscoverySpi(discoverySpi());
    ic.setCacheConfiguration(cacheConfiguration());
    return ic;
}

public CacheConfiguration cacheConfiguration() {
    CacheConfiguration cc = new CacheConfiguration();
    cc.setCacheMode(CacheMode.REPLICATED);
    return cc;
}

public int[] getEventTypes() {
    int[] eventTypes = {
            EventType.EVT_TASK_STARTED,
            EventType.EVT_TASK_FINISHED,
            EventType.EVT_TASK_FAILED,
            EventType.EVT_TASK_TIMEDOUT,
            EventType.EVT_TASK_SESSION_ATTR_SET,
            EventType.EVT_TASK_REDUCED,
            EventType.EVT_CACHE_OBJECT_PUT,
            EventType.EVT_CACHE_OBJECT_READ,
            EventType.EVT_CACHE_OBJECT_REMOVED,
            EventType.EVT_CLIENT_NODE_RECONNECTED,
            EventType.EVT_CACHE_OBJECT_EXPIRED
    };

    return eventTypes;
}

@Bean
public TcpDiscoverySpi discoverySpi() {
    TcpDiscoverySpi spi = new TcpDiscoverySpi();
    spi.setIpFinder(tdif());
    return spi;
}

@Bean
public TcpDiscoveryMulticastIpFinder tdif() {
    TcpDiscoveryMulticastIpFinder finder = new TcpDiscoveryMulticastIpFinder();
    List<String> addresses = new ArrayList<>();
    addresses.add("127.0.0.1:47500..47509");
    finder.setAddresses(addresses);
    return finder;
}
}

Solution

  • Client nodes don't store data of distributed cache (replicated or partitioned). You heave only one replica because you have only one server node. Just start more server nodes and you'll see that cache is replicated over all server nodes.