Search code examples
kuberneteskubectlrbac

How do you set up kubernetes RBAC resources so that pods can access the API via a client?


Problem

I have a simple RBAC configuration to access the Kubernetes API in-cluster. However I am getting what appears to be conflicting information from kubectl. After deploying the manifest, it appears that RBAC is set up properly.

$ kubectl exec -ti pod/controller -- kubectl auth can-i get namespaces
Warning: resource 'namespaces' is not namespace scoped
yes

However, actually making the request yields a permission error

$ kubectl exec -ti pod/controller -- kubectl get namespaces
Error from server (Forbidden): namespaces is forbidden: User "system:serviceaccount:default:controller" cannot list resource "namespaces" in API group "" at the cluster scope
command terminated with exit code 1

Manifest

apiVersion: 'v1'
kind: 'ServiceAccount'
metadata:
  name: 'controller'
---

apiVersion: 'rbac.authorization.k8s.io/v1'
kind: 'Role'
metadata:
  name: 'read-namespaces'
rules:
  - apiGroups:
      - ''
    resources:
      - 'namespaces'
    verbs:
      - 'get'
      - 'watch'
      - 'list'
---

apiVersion: 'rbac.authorization.k8s.io/v1'
kind: 'RoleBinding'
metadata:
  name: 'read-namespaces'
roleRef:
  apiGroup: ''
  kind: 'Role'
  name: 'read-namespaces'
subjects:
  - kind: 'ServiceAccount'
    name: 'controller'
---

apiVersion: 'v1'
kind: 'Pod'
metadata:
  name: 'controller'
  labels:
    'app': 'controller'
spec:
  containers:
    - name: 'kubectl'
      image: 'bitnami/kubectl:latest'
      imagePullPolicy: 'Always'
      command:
        - 'sleep'
        - '3600'
  serviceAccountName: 'controller'
---

Other Info

I've tried kubectl auth reconcile -f manifest.yaml as well as kubectl apply -f manifest.yaml and the results are the same.

I've also set "read-namespaces" RoleBinding.subjects[0].namespace to the proper namespace ("default" in this case). No change in output.


Solution

  • Namespace is a cluster scoped resource. So you need a ClusterRole and a ClusterRoleBinding.

    apiVersion: 'rbac.authorization.k8s.io/v1'
    kind: 'ClusterRole'
    metadata:
      name: 'read-namespaces'
    rules:
      - apiGroups:
          - ''
        resources:
          - 'namespaces'
        verbs:
          - 'get'
          - 'watch'
          - 'list'
    ---
    
    apiVersion: 'rbac.authorization.k8s.io/v1'
    kind: 'ClusterRoleBinding'
    metadata:
      name: 'read-namespaces'
    roleRef:
      apiGroup: 'rbac.authorization.k8s.io'
      kind: 'ClusterRole'
      name: 'read-namespaces'
    subjects:
      - kind: 'ServiceAccount'
        name: 'controller'
    ---