Search code examples
node.jspostgresqlknex.js

How can I instrument and log my KnexJS transactions?


I have a serious problem in production causing the application to become unresponsive and output the following error:

Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?

A running hypothesis is some operations are holding onto long-running Knex transactions. Enough of them to reach the pool size, basically.

  1. Is there a way to query the KnexJS API for how many pool connections are in use at any one time? Unfortunately since KnexJS occupies the max pool settings from the config, it can be hard to know how many are actually in use. From the postgres end, it seems like KnexJS is idling on all of its connections when they are not in use.
  2. Is there a good way to instrument Knex transaction and transacting with some kind of middleware or hook? Another useful thing is to log the callstack of any transaction (or any longer than, say, 7 seconds). One challenge is I have calls to Knex transaction and transacting throughout my project. Maybe it's a long shot.

Any advice is greatly appreciated.


System Information

  • KnexJS version: 0.12.6 (we will update in the next month)
  • Database + version: Postgres 9.6
  • OS: Heroku Linux (Ubuntu?)

Solution

  • Easiest was to see whats happening on connection pool level is to run knex with DEBUG=knex:* environment variable set, which will print quite a lot debug info whats happening inside knex. Those logs shows for example when connections are fetched from pool and returned to there and every ran query too.

    There are couple of global events that you can use to hookup to every query, but there is not any for hooking to transactions. Here is related question where I have written some example code how to actually measure transaction durations with query hooks though: Tracking DB querying time - Bookshelf/knex It probably leaks some memory, so its not very production ready solution, but for your debugging purposes it might be helpful.