Guava ratelimiter doesn't work when timeout set to >= 1 second

Question

I am using Guava RateLimiter and have been creating ratelimiter within my code like shown below.

public class RateLimitedCallable<T> implements Callable<T> {

    @Override
    public T call() {
        Boolean permitAcquired = RateLimitTest.rateLimiter.tryAcquire(1, 1000,
            TimeUnit.MILLISECONDS);

        if (permitAcquired) {
           // do stuff
        } else {
            throw new RuntimeException("Permit was not granted by RateLimiter");
        }
    }
}

public class RateLimitTest {

    public static final RateLimiter rateLimiter = RateLimiter.create(1.0);

    public void test() {
        RateLimitedCallable<String> callable = new RateLimitedCallable<>();
        callable.call();
        callable.call();
        callable.call();
        callable.call();
    }

    public static void main(String[] args) {
        RateLimitTest limiterTest = new RateLimitTest();
        limiterTest.test();
    } 

}

RuntimeException never gets thrown. However, if I change the timeout to values below 1000 ms, for eg: -

Boolean permitAcquired = RateLimitTest.rateLimiter.tryAcquire(1, 900, TimeUnit.MILLISECONDS);

I do see the RunTimeException, which means the ratelimiter works as expected. I don't understand why the ratelimiter doesn't enforce limits when timeout period is greater than equal to 1000ms. Am I doing something wrong?

Answer 1

First, it's good to remember what tryAcquire does (emphasis mine):

Acquires the given number of permits from this RateLimiter if it can be obtained without exceeding the specified timeout, or returns false immediately (without waiting) if the permits would not have been granted before the timeout expired.

In your single thread example, it's normal that it never throws any exception because each call waits roughly one second before it gets the permits. So here's what happens in your code:

1. The first call knows that it can get the permit immediately. So it acquires immediately the permit.
2. After the first call is fully done, the second call knows that it can get the permit if it waits. So it waits for ~1s and acquires the permit.
3. After the second call is fully done, the third call knows that it can get the permit if it waits. So it waits for ~1s and acquires the permit.
4. After the third call is fully done, the fourth call knows that it can get the permit if it waits. So it waits for ~1s and acquires the permit
5. End of program.

Now try using this in a multithreaded example, and you'll start seeing several failures and several successes. Because they all want to acquire the permit at the same time.

1. The first to acquire is happy.
2. Then the second knows if it waits ~1 second, it could get it, so it waits until it gets it.
3. The third and fourth see already 2 calls in the queue and know that they would have to wait for 2 seconds before getting the permit. So they give up because 2 seconds is greater than the timeout of 1 second that you set.

So basically, just use a multithreaded environment to test that it's going to happen.

  @Test
  void test() {
    var rateLimiter = RateLimiter.create(1.0);
    var stopwatch = Stopwatch.createStarted();
    var executor = Executors.newFixedThreadPool(4);
    for (var i = 0; i < 4; i++) {
      executor.submit(() -> {
        if (rateLimiter.tryAcquire(1, 1000, TimeUnit.MILLISECONDS)) {
          System.out.printf("Acquired after %s%n", stopwatch);
        } else {
          System.out.printf("Gave up trying to acquire after %s%n", stopwatch);
        }
      });
    }
    executor.shutdown();
    try {
      if (!executor.awaitTermination(5000, TimeUnit.MILLISECONDS)) {
        executor.shutdownNow();
      }
    } catch (InterruptedException e) {
      executor.shutdownNow();
    }
  }

This results in

Acquired after 12.76 ms
Gave up trying to acquire after 12.41 ms
Gave up trying to acquire after 12.43 ms
Acquired after 1.004 s