Postgres connection refused (via CloudSQL proxy) when doing rolling update Kubernetes

Question

When I do a rolling update, I get exceptions from Sentry saying:

DatabaseError('server closed the connection unexpectedly. This probably means the server terminated abnormally before or while processing the request.',...)

I have two containers running inside each Pod, my app container and a cloudsql-proxy container, which the app container uses to communicate to Cloud SQL.

Is there a way to make sure that my app container goes down first during the 30 seconds of grace period (terminationGracePeriodSeconds)?

In other words, I want to drain the connections and have all the current requests finish before the cloudsql-proxy is taken out.

It would be ideal if I could specify that the app container be taken down first during the 30 seconds of grace period, and then the cloudsql-proxy.

Answer 1

This discussion suggests setting the “terminationGracePeriodSeconds” or the "PreStop hook" in the manifest.

Another idea that could work is running the two containers in different pods to allow granular control over the rolling update. You might also want to consider using Init Containers on your deployment to allow the proxy to be ready before your app container.