Search code examples
kubernetesopenshiftkubernetes-helm

Helm does not upgrade rolebindings


I'm writing a helm chart to help us provision namespaces for different projets. The chart is taking care of creating the namespace, serviceaccounts, limits, quotas, volumes and rolebindings.

It's working pretty good so far, except for one point ...

Helm does not seem to update rolebindings if you run a helm upgrade.

We want to be able to reapply the chart regulary in order to correct any divergence from the reference point.

Actually, what helm upgrade do is creating the correct rolebinding if it doen't exist, but in the case the rolebinding is already present, but with different user/group subjects, it does NOT correct it.

I was able to enforce this by specifiying helm upgrade --force. In this case the rolebinding is being recreated, however, I can't use this technique : the --force option when used on a namespace recreated the userid of the namespace (sa.scc.uid-range) and then any NFS mounted volume have to be manually "chown" to the new UID ...

I've tried to use the annotation "helm.sh/resource-policy=keep" on the namespace but that does not help.

I'm stuck with these two options :

  • use "--force", in that case the rolebindings are recreated accordingly, but namespace UID change which leads to NFS permissions issues
  • not use "--force" and the live with the fact that the rolebinding defined in my helm templates may not be the one I need, they can deviate.

A solution would be to split the chart in two parts : one taking care of the namespace creation, and one taking care of the k8s objects inside, but I don't like this that much.

Maybe somebody has a better idea ?

---
{{- range .Values.evsEnv }}
apiVersion: authorization.openshift.io/v1
kind: RoleBinding
metadata:
  name: evs-rb-admin
  namespace: {{ $.Values.evsName }}-{{ .name }}
roleRef:
  name: admin
subjects:
  {{- range $.Values.evsGroupAdmin }}
  - kind: Group
    name: {{ . | quote }}
  {{- end }}
  {{- range $.Values.evsUserAdmin }}
  - kind: User
    name: {{ . | quote }}
  {{- end }}
  - kind: User
    name: {{ $.Values.evsRequester | quote }}
---
{- end }}

Solution

  • @titou10 was almost right, but you have to change the name instead. In my case, I tried to change roleRef.name, which cannot be patched also.

    Changing the name will force HELM to create a new object, and the old one will be removed.

    Template:

        kind: RoleBinding
        apiVersion: rbac.authorization.k8s.io/v1
        metadata:
          name: '{{ .Values.group }}-{{ .Values.role }}'
          namespace: namespace-name
        subjects:
          - kind: Group
            apiGroup: rbac.authorization.k8s.io
            name: '{{ .Values.group }}'
        roleRef:
          apiGroup: rbac.authorization.k8s.io
          kind: ClusterRole
          name: '{{ .Values.role }}'
    

    Changing .Values.role from edit to admin:

        $ helm upgrade 'role-binding' --install --debug -f values.yaml .
        history.go:56: [debug] getting history for release role-binding
        upgrade.go:153: [debug] preparing upgrade for role-binding
        upgrade.go:161: [debug] performing update for role-binding
        upgrade.go:354: [debug] creating upgraded release for role-binding
        client.go:393: [debug] checking 1 resources for changes
        client.go:414: [debug] Created a new RoleBinding called "GROUP-NAME-admin" in namespace-name
        
        client.go:442: [debug] Deleting RoleBinding "GROUP-NAME-edit" in namespace namespace-name...
        upgrade.go:169: [debug] updating status for upgraded release for role-binding
        Release "role-binding" has been upgraded. Happy Helming!
    

    Random also should work:

        metadata:
            name: "evs-rb-admin-{{ randAlphaNum 8 }}"
    

    Or you can use something more sophisticated:

        metadata:
            name: "evs-rb-admin-{{ toJson .Values.something | sha256sum | trunc 6 }}"