Search code examples
c++postgresqllibpq

Where is query pipelining in libpq?


I'm developing a high load server application using C++ and libpq (libpqxx is very unstable). I use async queries in threads to improve performance. But I've discovered that, for example, PQsendQuery("SELECT 1;SELECT 1;") works faster then executing PQsendQuery("SELECT 1") two times.

Is there any way to call bunch prepared of statements or another way to pipeline or use batch mode?

EDITED: Summary: How to get best performance in libpq?


Solution

  • libpq supports support query pipelining in PostgreSQL 14. You can use it on old PostgreSQL versions so long as you're using a new libpq. See commit acb7e4eb6b.

    I wrote a patch for it in 2016 as a bit of a fire-and-forget weekend hack Thanks to the efforts of many others, it has been turned into a mature feature and merged into PostgreSQL 14.

    The documentation for PostgreSQL 14 discusses pipeline mode.

    Note that PgJDBC has supported batch mode for many years using the standard JDBC batch interfaces. For many applications you can achieve similar performance improvements using PgJDBC batches.

    psycopg2 is currently adding or has added libpq-based adding batch and pipeline support.