Search code examples
ignite

Ignite Cache entries not evicted if inserted via SQL


Looks like the data is not getting evicted from the Ignite cache in spite of the expiration time set into the configuration. I notice that this issue happens when I use SQL to insert data into the cache table.

 emplCache.query(new SqlFieldsQuery(
   "CREATE TABLE Employee (id LONG PRIMARY KEY, firstName VARCHAR, lastName VARCHAR, 
    salary DECIMAL, gender VARCHAR)")).getAll();
 SqlFieldsQuery insertqry = new SqlFieldsQuery("INSERT INTO Employee (id, firstName,
    lastName, salary, gender) values (?, ?, ?, ?, ?)");
 emplCache.query(insertqry.setArgs(Long.toString(count), words[1], words[2],
    words[3], words[4])).getAll();

This was how I configured the expiry :

CacheConfiguration<?, ?> cfg = new CacheConfiguration<>(cacheName);
    cfg.setCacheMode(CacheMode.PARTITIONED);
    cfg.setName(cacheName);
    cfg.setSqlSchema("PUBLIC");
    cfg.setAtomicityMode(CacheAtomicityMode.ATOMIC);
    cfg.setExpiryPolicyFactory(CreatedExpiryPolicy.factoryOf(new Duration(SECONDS, 1)));

However, I do not see this issue if I use the dataStreamer to insert the data.

IgniteDataStreamer<Integer, Employee> stmr = ignite.dataStreamer(cfg.getName())
 stmr.addData(id, emp);

Have I missed some configuration setting?

[PS] Tried the following after Evgenii's answer:

// This is a util method that returns a cache config for the given cache name and the expiry time in seconds
        CacheConfiguration<?, ?> cfg = CacheConfig.getCacheConfig("emplCache", 1);
        IgniteCache<?, ?> emplCache = ignite.getOrCreateCache(cfg);
        emplCache.query(new SqlFieldsQuery(
                    "CREATE TABLE Employee (id LONG PRIMARY KEY, firstName VARCHAR, lastName VARCHAR, salary DECIMAL, gender VARCHAR)"
                    + "WITH \"template=emplCache\" ")).getAll();

Now i get this exception that the cache doesn't exist. But I am trying to create the table after creating the cache:

 Exception in thread "main" javax.cache.CacheException: class org.apache.ignite.internal.processors.query.IgniteSQLException: Cache doesn't exist: emplCache
at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:807)
at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:765)
at com.demo.ignite.svc.CsvStreamer.main(CsvStreamer.java:33)
Caused by: class org.apache.ignite.internal.processors.query.IgniteSQLException: Cache doesn't exist: emplCache
at org.apache.ignite.internal.processors.query.h2.ddl.DdlStatementsProcessor.convert(DdlStatementsProcessor.java:277)
at org.apache.ignite.internal.processors.query.h2.ddl.DdlStatementsProcessor.runDdlStatement(DdlStatementsProcessor.java:221)
at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.queryDistributedSqlFields(IgniteH2Indexing.java:1331)
at org.apache.ignite.internal.processors.query.GridQueryProcessor$5.applyx(GridQueryProcessor.java:1815)
at org.apache.ignite.internal.processors.query.GridQueryProcessor$5.applyx(GridQueryProcessor.java:1813)
at org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2293)
at org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:1820)
at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:795)

Solution

  • Table Employee will be created in another cache(that will be creted for it), thats why configured ExpiryPolicy will not be used.

    To configure this new cache add "WITH \"template=CACHE_NAME\""
    

    where CACHE_NAME is name of cache(or template) in your configuration. In your case it's cacheName;

    For example, change your creation script to:

    emplCache.query(new SqlFieldsQuery(
                "CREATE TABLE Employee (id LONG PRIMARY KEY, firstName VARCHAR, lastName VARCHAR, salary DECIMAL, gender VARCHAR) " +
                    "WITH \"template=employee\" ")).getAll();
    

    Here are link to doc

    If you configure cache from java, you will need to add this cache with name emplCache to Ignite with:

        ignite.addCacheConfiguration(cfg);