Search code examples
tcpconnectiondetectionc3p0

C3P0 half open connections closing


In my application am using C3P0 to connect to DB server. Here the DB server is running behind the firewall.

The firewall is misbehaving sometimes due to which the TCP connections are becoming half-open and C3P0 thinks these are valid connections and trying to fire the queries. The C3P0 helper threads are taking these half open queries and application threads are starving for the connections.

`"com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1" daemon prio=10 tid=0x00007fec0865d000 nid=0xc1fe runnable [0x00007fec4cb66000]
       java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:152)
        at java.net.SocketInputStream.read(SocketInputStream.java:122)
        at com.ingres.gcf.dam.IoBuff.fillBuffer(Unknown Source)
        at com.ingres.gcf.dam.IoBuff.next(Unknown Source)
        - locked <0x0000000782455468> (a com.ingres.gcf.dam.InBuff)
        at com.ingres.gcf.dam.InBuff.receive(Unknown Source)
        at com.ingres.gcf.dam.MsgIn.receive(Unknown Source)
        at com.ingres.gcf.dam.MsgConn.receive(Unknown Source)
        at com.ingres.gcf.jdbc.DrvObj.readResults(Unknown Source)
        at com.ingres.gcf.jdbc.JdbcConn.connect(Unknown Source)
        at com.ingres.gcf.jdbc.JdbcConn.<init>(Unknown Source)
        at com.ingres.gcf.jdbc.JdbcDrv.connect(Unknown Source)
        at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:81)
        - locked <0x0000000780025b10> (a com.mchange.v2.c3p0.DriverManagerDataSource)
        at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:96)
        - locked <0x0000000780025b70> (a com.mchange.v2.c3p0.WrapperConnectionPoolDataSource)
        at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1.acquireResource(C3P0PooledConnectionPool.java:89)
        at com.mchange.v2.resourcepool.BasicResourcePool.acquireUntil(BasicResourcePool.java:665)
        at com.mchange.v2.resourcepool.BasicResourcePool.access$500(BasicResourcePool.java:32)
        at com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask.run(BasicResourcePool.java:1206)
        at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:368)
    `

Is there any work around can be possible to overcome this situation?

The database used is VECTORWISE database.


Solution

  • If socket reads, from the perspective of the application, are hanging indefinitely -- neither succeeding nor failing with an Exception -- the only thing c3p0 can really do about it is time them out and try to interrupt them.

    The failure you show above isn't in a query, but in Connection acquisition. You can try setting maxAdministrativeTaskTime to time this out. Obviously, if c3p0 can't fetch Connections, attempts to use your DataSource will still fail. But they should (eventually, after a lot of timeouts, see acquireRetryAttempts and acquireRetryDelay) break explicitly, with Exceptions to clients, rather than just waiting forever

    (This depends upon the susceptibility of the hangs to Thread.interrupt(). If they can't be interrupted maxAdministrativeTaskTime can't help very much.)