Search code examples
kuberneteskubernetes-helm

Helm upgrade times out and fails to get configmaps


After months of using Helm (version 2.6.2) to deploy services in kubernetes, we've started experiencing random errors while performing the --upgrade command.

Most of the time the upgrade times out, in other situations instead it looks like a network issue occurs, with errors like getsockopt: connection refused or TLS handshake timeout.

Sometimes we've also seen the server cannot complete the requested operation at this time, try again later (get configmaps).

We're using Helm to deploy several versions a day of our services to our CI environment, and the instability of the deployment process that started to creep in is affecting our productivity.

Any idea what I should look for to restore the --upgrade command to a reliable state?


Solution

  • Upgrading to Helm 2.7.0 and using --history-max solved the issue for me, so the problem must have been related to the fact that old config maps were not cleared up by tiller, and over time they piled up until tiller started to struggle to make sense of them.

    More information about it here.