I am submitting Callable objects to a ThreadPoolExecutor and they seem to be sticking around in memory.
Looking at the heap dump with the MAT tool for Eclipse see that the Callable objects are being referenced by a FutureTask$Sync's callable variable. That FutureTask$Sync is referenced by a FutureTask's sync variable. That FutureTask is referenced by the FutureTask$Sync's this$0 variable.
I have read around about this (here, here, and on SO) and it seems like the FutureTask that the callable is wrapped in upon the ThreadPoolExecutor's submit() holds a reference to the callable forever.
What I am confused about is how to ensure that the FutureTask gets garbage collected so it doesn't continue to hold the callable in memory, and hold anything the callable might be holding in memory?
Just to give more details about my particular situation, I am trying to implement the ThreadPoolExecutor in a way that allows all of the submitted tasks to be canceled if needed. I have tried several different methods I found on SO and elsewhere, such as completely shutting the executor down (with shutdown(), shutdownNow() etc) and also keeping a list of the futures return by submit() and calling cancel on all them and then clearing the list of futures. Ideally I would like not to have to shut it down, and just cancel() and clear out when needed.
All of these methods don't seem to make a difference. If I submit a callable to the pool, there is a good chance it will end up sticking around.
What am I doing wrong?
Thanks.
Edit:
As requested, here is the constructor for the ThreadPoolExecutor.
public ThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
}
After further testing I can see that if I let the tasks that have been submitted to the ThreadPoolExecutor finish, then there is no leak. If I try to cancel them in anyway such as:
shutdownNow()
Or saving a reference to the future and calling cancel on it later:
Future referenceToCancelLater = submit(task);
...
referenceToCancelLater.cancel(false);
Or by removing them from the queue with methods like:
getQueue.drainTo(someList)
or
getQueue.clear()
or Looping through saved references to the futures and calling:
getQueue.remove(task)
Any of those cases causes the FutureTask to stick around as described above.
So the real question in all of this is how to I properly cancel or remove items from a ThreadPoolExecutor so that the FutureTask is garbage collected and not leaked forever?
I couldn't get anything to work so I came up with the following solution. Here is a rough overview: I created an array in the ThreadPoolExecutor that kept track of the runnables that were in the queue. Then when I needed to cancel the queue, I looped through and called a cancel method on each of the runnables. I my case, all of these runnables were a custom class I created and their cancel method simply set a cancelled flag. When the queue brought up the next one to process, in the run of the runnable it would see it was cancelled and skip the actual work.
So all of the runnables then just get flushed out quickly one by one as it sees it was cancelled.
Probably not the greatest solution, but it works for me and it doesn't leak memory.