I am building an application which includes a scraper of a public API. I want to limit the rate of my requests but not with a fixed rate, but a dynamic randomised rate which changes in-between requests and falls in a certain range, i.e. I can set the minimum and maximum delay between requests.
I also don't want to block the current thread.
I've researched Guava Ratelimiter, RatelimitJ and Token bucket algorithm but looking at the docs, I don't see how any of those things can achieve what I want.
Example of the desired effect on my requests given a rate limit range of 250-350ms:
You don't need to look outside the JDK. You can use java.util.concurrent.ScheduledExecutorService to make the call after a defined delay and in a separate thread. To use this you will need to implement java.util.concurrent.Callable or java.lang.Runnable for the code that does the API call.
ScheduledExecutorService service = Executors.newScheduledThreadPool(1);
Callable apiCaller = ...;
To make the call so that it runs after a defined delay you submit it as follows:
service.schedule(apiCaller, delay, TimeUnit.MILLISECONDS);
To get a random delay you just need to use java.util.Random.
Random random = new Random();
Random can provide you with a bounded integer. So if you pass it the acceptable range and then add then add the minimum you will get a random number between the minimum and maximum you want.
int minimumDelay = 250;
int maximumDelay = 350;
int delay = random.nextInt(maximumDelay - minimumDelay) + minimumDelay;