Search code examples
javamemory-leaksout-of-memoryvert.xjooq

Memory leak of org.postgresql.jdbc.PgResultSet instance and io.vertx.core.impl.VertxThread

enter image description hereI am using vert.x and jooq in my application. I have one method which fetch data from database. this method is called continuously but one after one (on the success of other) . After some time I am getting heap out of memory issue. I checked heap dump with Eclipse memory analyzer and it shows one object has memory leak i.e org.postgresql.jdbc.PgResultSet. It shows two problem. 2nd is The thread io.vertx.core.impl.VertxThread @ 0x680c5eb80 keeps local variables with total size 2,246,312,920 (38.58%) bytes. but I as per hint given, both are related.

following is stacktrace provided by eclipse tool

[![at org.postgresql.core.UTF8Encoding.decode(\[BII)Ljava/lang/String; (UTF8Encoding.java:156)
  at org.postgresql.core.Encoding.decode(\[B)Ljava/lang/String; (Encoding.java:215)
  at org.postgresql.jdbc.PgResultSet.getString(I)Ljava/lang/String; (PgResultSet.java:1926)
  at com.zaxxer.hikari.pool.HikariProxyResultSet.getString(I)Ljava/lang/String; (Unknown Source)
  at org.jooq.tools.jdbc.DefaultResultSet.getString(I)Ljava/lang/String; (DefaultResultSet.java:114)
  at org.jooq.impl.CursorImpl$CursorResultSet.getString(I)Ljava/lang/String; (CursorImpl.java:938)
  at org.jooq.impl.DefaultBinding$DefaultStringBinding.get0(Lorg/jooq/BindingGetResultSetContext;)Ljava/lang/String; (DefaultBinding.java:3334)
  at org.jooq.impl.DefaultBinding$DefaultStringBinding.get0(Lorg/jooq/BindingGetResultSetContext;)Ljava/lang/Object; (DefaultBinding.java:3270)
  at org.jooq.impl.DefaultBinding$AbstractBinding.get(Lorg/jooq/BindingGetResultSetContext;)V (DefaultBinding.java:774)
  at org.jooq.impl.CursorImpl$CursorIterator$CursorRecordInitialiser.setValue(Lorg/jooq/impl/AbstractRecord;Lorg/jooq/Field;I)V (CursorImpl.java:1771)
  at org.jooq.impl.CursorImpl$CursorIterator$CursorRecordInitialiser.operate(Lorg/jooq/impl/AbstractRecord;)Lorg/jooq/impl/AbstractRecord; (CursorImpl.java:1740)
  at org.jooq.impl.CursorImpl$CursorIterator$CursorRecordInitialiser.operate(Lorg/jooq/Record;)Lorg/jooq/Record; (CursorImpl.java:1705)
  at org.jooq.impl.RecordDelegate.operate(Lorg/jooq/impl/RecordOperation;)Lorg/jooq/Record; (RecordDelegate.java:125)
  at org.jooq.impl.CursorImpl$CursorIterator.fetchNext()Lorg/jooq/Record; (CursorImpl.java:1669)
  at org.jooq.impl.CursorImpl$CursorIterator.hasNext()Z (CursorImpl.java:1636)
  at org.jooq.impl.CursorImpl.fetchNext(I)Lorg/jooq/Result; (CursorImpl.java:408)
  at org.jooq.impl.CursorImpl.fetch(I)Lorg/jooq/Result; (CursorImpl.java:394)
  at org.jooq.impl.CursorImpl.fetch()Lorg/jooq/Result; (CursorImpl.java:301)
  at org.jooq.impl.AbstractResultQuery.execute(Lorg/jooq/ExecuteContext;Lorg/jooq/ExecuteListener;)I (AbstractResultQuery.java:297)
  at org.jooq.impl.AbstractQuery.execute()I (AbstractQuery.java:350)
  at org.jooq.impl.AbstractResultQuery.fetch()Lorg/jooq/Result; (AbstractResultQuery.java:323)
  at org.jooq.impl.AbstractResultQuery.fetchInto(Ljava/lang/Class;)Ljava/util/List; (AbstractResultQuery.java:1440)
  at org.jooq.impl.SelectImpl.fetchInto(Ljava/lang/Class;)Ljava/util/List; (SelectImpl.java:3741)
  at com.module.BuildConfigCacheManagerImpl.lambda$getAllConfigurations$3(Ljava/util/List;Lorg/jooq/DSLContext;)Ljava/util/List; (BuildConfigCacheManagerImpl.java:99)
  at com.module.BuildConfigCacheManagerImpl$$Lambda$501.apply(Ljava/lang/Object;)Ljava/lang/Object; (Unknown Source)
  at io.github.jklingsporn.vertx.jooq.classic.jdbc.JDBCClassicGenericQueryExecutor.lambda$executeAny$0(Ljava/util/function/Function;Lio/vertx/core/Future;)V (JDBCClassicGenericQueryExecutor.java:30)
  at io.github.jklingsporn.vertx.jooq.classic.jdbc.JDBCClassicGenericQueryExecutor$$Lambda$486.handle(Ljava/lang/Object;)V (Unknown Source)
  at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$2(Lio/vertx/core/spi/metrics/PoolMetrics;Ljava/lang/Object;Lio/vertx/core/Handler;Lio/vertx/core/Handler;)V (ContextImpl.java:272)
  at io.vertx.core.impl.ContextImpl$$Lambda$439.run()V (Unknown Source)
  at io.vertx.core.impl.TaskQueue.run()V (TaskQueue.java:76)
  at io.vertx.core.impl.TaskQueue$$Lambda$90.run()V (Unknown Source)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V (ThreadPoolExecutor.java:1128)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run()V (ThreadPoolExecutor.java:628)
  at io.netty.util.concurrent.FastThreadLocalRunnable.run()V (FastThreadLocalRunnable.java:30)
  at java.lang.Thread.run()V (Thread.java:834)][1]][1]

Can anyone please help me here whats going wrong here.

Solution

  • A "memory leak" where a JDBC result set is involved is likely due to one of three reasons:

    1. A huge amount of data is fetched into the client. This would not be a leak, but just inefficient memory usage. It could be solved by specifying a JDBC fetch size via ResultQuery.fetchSize()
    2. You're fetching a huge object into the client. The exception is triggered by a getString() call. Perhaps that particular string is simply way too big to be fetched into memory. You could use a data type binding to implement large object streaming logic on top of jOOQ.
    3. The connection pool produces proxied connections that do not correctly close depending resources, such as result sets. In this case, there might be a misconfiguration in the connection pool, or a bug.