How to hit 1000 endpoints using multi threading in groovy?

Question

I have a requirement to hit endpoints more than 1000 times to fetch some data from website. So i read some tutorials to use Multi Threading to achieving it. But at a time i want to use only 13 threads on same method.

So basically i am using ExecutorService to run 13 threads at one time:

ExecutorService threadPool = Executors.newFixedThreadPool(13);
for (int itLocation = 0; itLocation < locationList.size(); itLocation++) {
  //some code like
  ScraperService obj = new ScraperService(threadName,url)
  threadPool.submit(obj);
}
threadPool.shutdown();

My Groovy Class named as ScraperService is implementing the Runnable interface.

@Override
        void run() {
            println("In run method...................")
            try {
                Thread.sleep(5000);
                someMethod()
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }

Problem:

My problem is that my ExecutorService.submit(obj) and ExecutorService.execute(obj) is not calling my run() method of Runnable interface.

In Groovy/Grails:

There is also a executor plugin Executor Plugin in grails but i didn't found any appropriate example how to use it.

Answer 1

First of all i think there is problem in groovy service class with @Transnational annotation which doesn't allow to call run() method of Runnable interface. If you'll remove the @Transnational then it will call the run() method. It happened also in my case. But i am not sure, there may be some another reason. You can directly use:

ExecutorService threadPool =  Executors.newFixedThreadPool(13)
threadPool.execute(new Runnable() {
                            @Override
                            void run() {
                                Thread.sleep(5000);
                                someMethod()
                            }
                        })

Extra (As i read your question)

If you are using multiple threads on same method then it can be complex as all threads will use the same local variables of that method which can occurs problems. It is better to use multiple threads for different different work.

But if you want to use same method for executing multiple threads then in my scenario it is better to use Executors.newSingleThreadExecutor().

ExecutorService threadPool = Executors.newSingleThreadExecutor();

newSingleThreadExecutor() calls a single thread so if you want to execute multiple tasks on it then it does not create multiple threads. Instead, it waits for one task to complete before starting the next task on the same thread.

Speed: newSingleThreadExecutor in comparison of multi threads will be slow but safer to use.