Java ThreadPoolExecutor Hangs while processing

Question

I am having a custom Thread Pool Executor

public class CustomTPExecutor extends ThreadPoolExecutor
{

     /* Constructor called from my processor */

     public CustomTPExecutor (int corePoolSize, int maxPoolSize, long keepAliveTime,
  TimeUnit timeUnit, BlockingQueue<Runnable> blockingQueue,
  ThreadFactory threadFactory,Processor processor)
      {
        super(corePoolSize, maxPoolSize, keepAliveTime, timeUnit, blockingQueue,threadFactory);
        this.processor = processor;
     }
     @Override
     protected void afterExecute(Runnable paramRunnable, Throwable paramThrowable) 
    {
      super.afterExecute(paramRunnable, paramThrowable);
      /* Notify the processor to get more records */
    }
}

In my processor, I will be creating instances for this customTPExecutor and submit more than 1000+ tasks to it. The maximum pool size is 5.

For submitting it to executor we use this.

while(iterating map events containing 1000+records) {
 CustomThread tt = new TxnTCustomThread(eventMap.get(key), key,maxDate,connPool,connPool.getConnection(),email);
  executor.submit(tt); 
}

After processing all the records we are calling the executor shutdown method.

One thing we are noticing is that, while the threads are executing it hangs for sometime in the middle of processing. It pauses for a minute or so and then resumes processing the records. I am not sure what is the reaosn for this.

I am assuming it is because of garbage collection not properly happening. I cannot get anything from the logs also to debug on this. During this process memory consumption from java is reduced drastically.

Any suggestions on how this issue could be resolved will be really helpful.

Answer 1

There's such a thing as "stop-the-world" GC, which roughly speaking means that the JVM decided it needs such a "deep clean" that it temporarily stops all of the threads to prevent references from changing. This is not desired and so the GC generally tries to avoid it in favor of doing GC in the background. It definitely can happen though, especially if you're using all of the cores for computation (which may tend to prevent background GC) and you're generating a lot of objects.

If you search online for "stop-the-world GC" you'll get some good hits.