Search code examples
pythonpython-3.xpostgresqlpsycopg2

How to fix 'child connection forced to terminate due to client_idle_limit:180 is reached'


I am working on a Python script which sends queries to a PostgreSQL database. It works just fine until it stops for 3 minutes for unknown reasons. I can't seem to identify why it stops, probably due to internet connection. After it continues the following exception is thrown:

unable to read data DETAIL: child connection forced to terminate due to client_idle_limit:180 is reached server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. Traceback (most recent call last): File "/scraper/Database.py", line 36, in fetch_one_iin cursor.execute(query) psycopg2.OperationalError: unable to read data DETAIL: child connection forced to terminate due to client_idle_limit:180 is reached server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request.

How can I avoid this exception?

I use psycopg2==2.9.3, Python 3.8.

I thought about sending 'SELECT 1;' query from time to time, but it seems like a silly workaround.

I tried adding the following code to every function connecting to database:

        if self.conn.closed != 0:
            self.logger.warning("Connection is closed. Reconnecting...")
            self.conn = psycopg2.connect(self.connections[0])

But it didn't work, exception was still there.


Solution

  • Turns out conn.closed only returns non-zero value if the connection had been closed from the client side, that's why my program wasn't reopening the connection as intended.

    Following the advice from @Adrian_Klaver I tried doing this: I removed this code:

      if self.conn.closed != 0:
            self.logger.warning("Connection is closed. Reconnecting...")
            self.conn = psycopg2.connect(self.connections[0])
    

    And had a separate connection for every single query, i.e. I opened a connection at the start of a method and closed it at the end. Now the problem is fixed.