Caffeine eviction by size seems to not work

Question

I am using caffeine cache.

I want to put it under size limitation but it does not work properly.

test 1:

Cache<String, String> cache = Caffeine.newBuilder()
                .maximumSize(3)
                .build();

        Cache<String, String> cache = Caffeine.newBuilder()
                .maximumSize(3)
                .build();

        for (int i = 1; i <= 10; i ++) {
            String val = String.valueOf(i);
            cache.put(val, val);
        }

        System.out.println("cache size: " + cache.estimatedSize() + ", cache keys: " + cache.asMap().values().stream().collect(Collectors.joining(",")));


result:   cache size: 10, cache keys: 1,2,10

another test: trying to get key and set max to 1

Cache<String, String> cache = Caffeine.newBuilder()
                .maximumSize(1)
                .build();

        for (int i = 1; i <= 10; i ++) {
            String val = String.valueOf(i);
            cache.put(val, val);

            if (i % 2 == 0) {
                cache.getIfPresent("5");
            }
        }

        System.out.println("cache size: " + cache.estimatedSize() + ", cache keys: " + cache.asMap().values().stream().collect(Collectors.joining(",")));


cache size: 10, cache keys: 2,3,4,5,6,7,8,9,10

last test : run 100 times, max size 1

 Cache<String, String> cache = Caffeine.newBuilder()
                .maximumSize(1)
                .build();

        for (int i = 1; i <= 100; i ++) {
            String val = String.valueOf(i);
            cache.put(val, val);

            if (i % 2 == 0) {
                cache.getIfPresent("5");
            }
        }

        System.out.println("cache size: " + cache.estimatedSize() + ", cache keys: " + cache.asMap().values().stream().collect(Collectors.joining(",")));

cache size: 99, cache keys: 96,97,99,19,23,58

can someone please help me understand this and how to make it work properly?

---

Thanks to Ben Manes, I added .executor(Runnable::run)

Now after doing this I do get only 3 items

 Cache<String, String> cache = Caffeine.newBuilder()
                .maximumSize(3)
                .executor(Runnable::run)
                .build();

        for (int i = 1; i <= 10; i ++) {
            String val = String.valueOf(i);
            cache.put(val, val);

            if (i % 2 == 0) {
                cache.getIfPresent("5");
            }
        }
        cache.cleanUp();
        System.out.println("cache size: " + cache.estimatedSize() + ", cache: " + CodecUtils.toJson(cache.asMap().values()));


cache size: 3, cache: ["3","9","10"]

1. wouldn't this block my thread?
2. why isn't key 5 in the cache as I have been using it several times?

Answer 1

By default the cache will perform some operations asynchronously, such as eviction and notifying a removal listener. This is to minimize request latencies, as the auxiliary work is not necessary for the request itself and the user supplied callbacks might be expensive.

The cache's own maintenance work is very cheap, so you can safely run it on the caller's thread if desired by using Caffeine.executor(Runnable::run). This will penalize the caller with additionally evicting the entry, but will not block other operations from occurring. This is due to the cache internally using multiple locks and operation buffers, so that it can schedule work when a lock is busy rather than block threads.

In regards to the size, this is because the entry is evicted prior to being retrieved so it doesn't build up frequency. A getIfPresent doesn't increase the frequency if the entry is absent, whereas get(key, /* loading function */) would because it is penalized to load the value on the miss. The eviction policy utilizes both recency and frequency in its decisions, so it may evict recent arrivals early as often "one-hit wonders", aka cache pollution.

If we take your code as is and output the cache's state we see this,

for (int i = 1; i <= 10; i++) {
  String val = String.valueOf(i);
  cache.put(val, val);
  System.out.println(val + " -> " + cache.asMap());
  if (i % 2 == 0) {
    cache.getIfPresent("5");
  }
}
cache.cleanUp();
System.out.println("cache size: " + cache.estimatedSize());

1 -> {1=1}
2 -> {1=1, 2=2}
3 -> {1=1, 2=2, 3=3}
4 -> {2=2, 3=3, 4=4}
5 -> {2=2, 3=3, 5=5}
6 -> {2=2, 3=3, 6=6}
7 -> {2=2, 3=3, 7=7}
8 -> {2=2, 3=3, 8=8}
9 -> {2=2, 3=3, 9=9}
10 -> {2=2, 3=3, 10=10}
cache size: 3

If we access key 5 on every iteration then it is retained,

for (int i = 1; i <= 10; i++) {
  String val = String.valueOf(i);
  cache.put(val, val);
  System.out.println(val + " -> " + cache.asMap());
  cache.getIfPresent("5");
}
cache.cleanUp();
System.out.println("cache size: " + cache.estimatedSize());

1 -> {1=1}
2 -> {1=1, 2=2}
3 -> {1=1, 2=2, 3=3}
4 -> {2=2, 3=3, 4=4}
5 -> {2=2, 3=3, 5=5}
6 -> {3=3, 5=5, 6=6}
7 -> {3=3, 5=5, 7=7}
8 -> {3=3, 5=5, 8=8}
9 -> {3=3, 5=5, 9=9}
10 -> {3=3, 5=5, 10=10}
cache size: 3