I am working on a Python script which sends queries to a PostgreSQL database. It works just fine until it stops for 3 minutes for unknown reasons. I can't seem to identify why it stops, probably due to internet connection. After it continues the following exception is thrown:
unable to read data DETAIL: child connection forced to terminate due to client_idle_limit:180 is reached server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. Traceback (most recent call last): File "/scraper/Database.py", line 36, in fetch_one_iin cursor.execute(query) psycopg2.OperationalError: unable to read data DETAIL: child connection forced to terminate due to client_idle_limit:180 is reached server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request.
How can I avoid this exception?
I use psycopg2==2.9.3, Python 3.8.
I thought about sending 'SELECT 1;' query from time to time, but it seems like a silly workaround.
I tried adding the following code to every function connecting to database:
if self.conn.closed != 0:
self.logger.warning("Connection is closed. Reconnecting...")
self.conn = psycopg2.connect(self.connections[0])
But it didn't work, exception was still there.
Turns out conn.closed only returns non-zero value if the connection had been closed from the client side, that's why my program wasn't reopening the connection as intended.
Following the advice from @Adrian_Klaver I tried doing this: I removed this code:
if self.conn.closed != 0:
self.logger.warning("Connection is closed. Reconnecting...")
self.conn = psycopg2.connect(self.connections[0])
And had a separate connection for every single query, i.e. I opened a connection at the start of a method and closed it at the end. Now the problem is fixed.