I am seeing a warning in logs that Connection has been abandoned, and it is followed by an error nested exception is java.sql.SQLException: SSL peer shut down incorrectly
Here is the stack trace:
[2019-09-04 22:42:04.243] - 10748 WARNING [Tomcat JDBC Pool Cleaner[414493378:1567654809228]] --- org.apache.tomcat.jdbc.pool.ConnectionPool: Connection has been abandoned PooledConnection[com.mysql.cj.jdbc.ConnectionImpl@4857dbcc]:java.lang.Exception
at org.apache.tomcat.jdbc.pool.ConnectionPool.getThreadDump(ConnectionPool.java:1107)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:811)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:655)
at org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:198)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:132)
at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:157)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:115)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:78)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:611)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:668)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:699)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:711)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:762)
at com.****.****.*****.dao.GetTransactionDaoImpl.****(GetTransactionDaoImpl.java:**)
at com.****.****.*****.dao.GetTransactionDaoImpl$$FastClassBySpringCGLIB$$3305e4f5.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
at com.****.****.****.dao.GetTransactionDaoImpl$$EnhancerBySpringCGLIB$$b182bbd4.****(<generated>)
at com.****.****.****.***.****Test.***(****Test.java:***)
at com.****.****.****.****.****Test.*****(****Test.java:336)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:74)
at org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:84)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:367)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:274)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:161)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
Tests run: 10, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 281.049 sec <<< FAILURE! - in ****.****.****.****.*****.****Test
****(***.***.***.****.****.****Test) Time elapsed: 213.963 sec <<< ERROR!
org.springframework.dao.TransientDataAccessResourceException: PreparedStatementCallback; SQL [*******]; SSL peer shut down incorrectly; nested exception is java.sql.SQLException: SSL peer shut down incorrectly
THis is the connection pool setting I am using:
driver-class-name: com.mysql.cj.jdbc.Driver
initial-size: 15
max-wait: 20000
max-active: 50
max-idle: 15
min-idle: 8
test-on-borrow: true
log-abandoned: true
remove-abandoned: true
remove-abandoned-timeout: 60
default-transaction-isolation: 2
This doesn't happen all the time but happens a lot. I have tried different setting for remove-abandoned-timeout but still, the same thing keeps happening.
Can anyone provide any insights into connection pooling?
I figured it out, One of the query was taking a ling time. I had to increase the remove-abandoned-timeout. IT was taking more than 60 seconds, and in the middle of it, it was removing the connection.