Search code examples
jenkinskuberneteskubernetes-helm

Kubernetes make changes to annotation to force update deployment


Hey I have a wider problem as when I update secrets in kubernetes they are not implemented in pods unless they are ugprades/reschedules or just re-deployed; I saw the other stackoverflow post about it but noone of the solutions fit me Update kubernetes secrets doesn't update running container env vars

Also so the in-app solution of python script on pod to update its secret automatically https://medium.com/analytics-vidhya/updating-secrets-from-a-kubernetes-pod-f3c7df51770d but it seems like a long shot and I came up with solution to adding annotation to deployment manifest - and hoping it would re-schedule pods everytime a helm chart would put a new timestamp in it - it does put it but it doesn't reschedule - any thought how to force that behaviour ?

apiVersion: apps/v1
kind: Deployment
metadata:
  name: xxx
  namespace: xxx
  labels: xxx
  annotations:
    lastUpdate: {{ now }}

also I dont feel like adding this patch command to ci/cd deployment, as its arbitraty and - well doesnt feel like right solution

kubectl patch deployment mydeployment -p '{"spec":{"template":{"spec":{"containers":[{"name":"mycontainer","env":[{"name":"RESTART_","value":"'$(date +%s)'"}]}]}}}}'

didn't anyone else find better solution to re-deploy pods on changed secrets ?


Solution

  • Kubernetes by itself does not do rolling update of a deployment automatically when a secret is changed. So there needs to a controller which will do that for you automatically. Take a look at Reloader which is a controller that watches if some change happens in ConfigMap and/or Secret; then perform a rolling upgrade on relevant DeploymentConfig, Deployment, Daemonset and Statefulset.

    Add reloader.stakater.com/auto annotation to the deployment with name xxx and have a ConfigMap called xxx-configmap or Secret called xxx-secret.

    This will discover deployments/daemonsets/statefulset automatically where xxx-configmap or xxx-secret is being used either via environment variable or from volume mount. And it will perform rolling upgrade on related pods when xxx-configmap or xxx-secret are updated

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: xxx
      namespace: xxx
      labels: xxx
      annotations:
        reloader.stakater.com/auto: "true"