I am using the MarkLogic Java client to load a massive amount of data. I am getting the following error after a while and then it just repeats.
23-May-2017 15:09:11.199 WARNING [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [easymetahub] appears to have started a thread named [pool-20-thread-12] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:105)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1078)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:760)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1524)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
java.lang.IllegalStateException: This instance has been stopped
at com.marklogic.client.datamovement.impl.WriteBatcherImpl.requireNotStopped(WriteBatcherImpl.java:347)
at com.marklogic.client.datamovement.impl.WriteBatcherImpl.add(WriteBatcherImpl.java:283)
at com.marklogic.client.datamovement.impl.WriteBatcherImpl.add(WriteBatcherImpl.java:267)
at com.easymetahub.HarvestJDBCData.doRoot(HarvestJDBCData.java:486)
at com.easymetahub.HarvestJDBCData.doSomething(HarvestJDBCData.java:243)
at com.easymetahub.HarvestJDBCData.processBatchSegment(HarvestJDBCData.java:146)
at com.easymetahub.HarvestJDBCDataServlet.doPost(HarvestJDBCDataServlet.java:33)
at com.easymetahub.HarvestJDBCDataServlet.doGet(HarvestJDBCDataServlet.java:45)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:232)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:105)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:1078)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:760)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1524)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
java.lang.IllegalStateException: This instance has been stopped
at com.marklogic.client.datamovement.impl.WriteBatcherImpl.requireNotStopped(WriteBatcherImpl.java:347)
at com.marklogic.client.datamovement.impl.WriteBatcherImpl.add(WriteBatcherImpl.java:283)
at com.marklogic.client.datamovement.impl.WriteBatcherImpl.add(WriteBatcherImpl.java:267)
at com.easymetahub.HarvestJDBCData.doRoot(HarvestJDBCData.java:486)
at com.easymetahub.HarvestJDBCData.doSomething(HarvestJDBCData.java:243)
at com.easymetahub.HarvestJDBCData.processBatchSegment(HarvestJDBCData.java:146)
at com.easymetahub.HarvestJDBCDataServlet.doPost(HarvestJDBCDataServlet.java:33)
at com.easymetahub.HarvestJDBCDataServlet.doGet(HarvestJDBCDataServlet.java:45)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:232)
I have the following code that I copied and modified from the example:
client = DatabaseClientFactory.newClient("localhost", 8000, "emh-entity-manager-content", new DatabaseClientFactory.DigestAuthContext("admin", "mladmin"));
DataMovementManager dmm = client.newDataMovementManager();
batcher = dmm.newWriteBatcher();
batcher.withBatchSize(5000)
.withThreadCount(30)
.onBatchSuccess(batch-> {
System.out.println(
batch.getTimestamp().getTime() +
" documents written: " +
batch.getJobWritesSoFar() +
" \t[" + sourceName + "] [" + start + "]");
})
.onBatchFailure((batch,throwable) -> {
System.out.println("Failure on " + sourceName);
throwable.printStackTrace();
});
dmm.startJob(batcher);
doSomething(harvestNode);
// Start any partial batches waiting for more input, then wait
// for all batches to complete. This call will block.
batcher.flushAndWait();
dmm.stopJob(batcher);
Where the doSomething calls
batcher.add(entityPath, getStringFromDocument(outDoc));
How can I avoid this error?
I don't see anything that you're doing wrong in your code sample. I'm guessing the stack trace you shared is from some different code. You normally wouldn't want to create a new DatabaseClient inside a method like HarvestJDBCDataServlet.doPost because you want your DatabaseClient instance to be shared for the life of the webapp because it contains the shared connection pool. The only way you would get the error you're seeing is if you call batcher.add after you've already called dmm.stopJob(batcher). Is this the only error in the logs or does another error precede this? In any case, if you think you've encountered a bug, please file an issue in github.