Caffeine LoadingCache - Eviction with Custom expiration policy

Using Caffeine 2.8.1 and Java 8. I have created LoadingCache<String, Boolean>. I am loading the cache with cache.putAll(getAllKeyValues()) where getAllKeyValues() returns a Map<String, Boolean>.

I have specified a CacheLoader to compute the value by calling keyExistsOnServer(key) method which calls an external service to get the value.

I have specified a Custom expiration policy where I want to evict the entry from cache in 10 minutes if the value is set as false while creating/updating the entry in the cache. And keep it as long as Long.MAX_VALUE if the value is true.

  private RemovalListener<String, Boolean> removeListener =
      (key, value, cause) ->
          logger.info(
              "Entry for Key={} with value={} was removed ({}) from cache", key, value, cause);
  private LoadingCache<String, Boolean> cache =
      Caffeine.newBuilder()
          .expireAfter(
              new Expiry<String, Boolean>() {
                private static final long EXPIRATION_TEN_MINS_IN_NANOSECONDS = 600000000000L;

                @Override
                public long expireAfterCreate(String key, Boolean value, long currentTime) {
                  // If value is false that means key does not exist, so set expiration to 10 mins
                  return value ? Long.MAX_VALUE : EXPIRATION_TEN_MINS_IN_NANOSECONDS;
                }

                @Override
                public long expireAfterUpdate(
                    String key, Boolean value, long currentTime, long currentDuration) {
                  // If value is changed from true to false then set expiration to 10 mins
                  return value ? Long.MAX_VALUE : EXPIRATION_TEN_MINS_IN_NANOSECONDS;
                }

                @Override
                public long expireAfterRead(
                    String key, Boolean value, long currentTime, long currentDuration) {
                  // Don't modify expiration time after read
                  return currentDuration;
                }
              })
          .recordStats()
          .removalListener(removeListener)
          .build(key -> keyExistsOnServer(key));

Question#1: Does my expiration policy looks correct based on what I want to achieve?

Question#2:

I do not see the RemovalListener getting called as per the eviction policy. This may be due the accrual of tasks for cleanup as stated in this github issue.

However, correctness of my code relies on the fact that once the expiration duration (10 minutes in case of false value) have passed for an entry and if we call cache.get(key) then it should not return the expired value from cache, but rather call the CacheLoader i.e. keyExistsOnServer(key) method to get the value. Can someone assert that this is how it will behave?

Here is what @Louis Wasserman has stated on the github issue but I not clear if this clarifies it:

Actually, what happens is that the get call itself discovers that the entry is expired and adds it to the queue of entries to be cleaned up

Question#3: What happens if there is RuntimeException thrown by CacheLoader that is keyExistsOnServer(key) method on calling cache.get(key)?

Question#4: Would cache.asMap() contain the entries which are expired but were not evicted due to accrual for cleanup?

Question#5: When I am logging the currentDuration in the expireAfterRead() method, it does not seem consistent with the elapsed time. i.e. if it was set to 600000000000 (10 mins) with an update for the value to false I am expecting that after 5 mins it should be 300000000000 but it was not. Why so?

Solution

Question#1: Does my expiration policy looks correct based on what I want to achieve?

Yes, this looks good. You might make your constant a Duration or use TimeUnit for the calculation, just to make it look prettier.

Question#2: I do not see the RemovalListener getting called as per the eviction policy.

This will occur when enough activity on the cache occurs. You can specify a Scheduler which will wake up and call cache.cleanUp for you, based on when the next entry is set to expire. For Java 9+ users, Java offers a built-in scheduling thread that you can leverage by adding Caffeine.scheduler(Scheduler.systemScheduler) when constructing the cache.

Once the expiration duration has passed for an entry and if we call cache.get(key) then it should not return the expired value from cache, but rather call the CacheLoader.

Correct. The cache will validate the entry on lookup and, if expired, load it anew. It will never return an expired entry.

Question#3: What happens if there is RuntimeException thrown by CacheLoader that is keyExistsOnServer(key) method on calling cache.get(key)?

The mapping will not be established and the exception will propagate to the caller. The next call to get(key) will try anew, which might fail again. It is your choice of how to be resilient towards this. Often not being so is a good answer, or sometimes caching that it failed is a better one.

Question#4: Would cache.asMap() contain the entries which are expired but were not evicted due to accrual for cleanup?

The cache will hold the entries but suppress them from being visible, e.g. by lookup or iteration. The only external indication is that size() since that counter is not updated until the expired entry is removed.

Question#5: When I am logging the currentDuration in the expireAfterRead() method, it does not seem consistent with the elapsed time. i.e. if it was set to 600000000000 (10 mins) with an update for the value to false I am expecting that after 5 mins it should be 300000000000 but it was not. Why so?

Please open an issue with a unit test and we can diagnose.