Search code examples
kubernetesazure-akskubernetes-secrets

How allow pod from default namespace, read secret from other namespace


In Azure Kubernetes I want have a pod with jenkins in defualt namespace, that needs read secret from my aplication workspace.

When I tried I get the next error:

Error from server (Forbidden): secrets "myapp-mongodb" is forbidden: User "system:serviceaccount:default:jenkinspod" cannot get resource "secrets" in API group "" in the namespace "myapp"

How I can bring access this jenkisn pod to read secrets in 'myapp' namespace


Solution

  • secret is a namespaced resource and can be accessed via proper rbac permissions. However any improper rbac permissions may lead to leakage.

    You must role bind the pod's associated service account. Here is a complete example. I have created a new service account for role binding in this example. However, you can use the default service account if you want.

    step-1: create a namespace called demo-namespace

    kubectl create ns demo-namespace
    

    step-2: create a secret in demo-namespace:

    kubectl create  secret generic  other-secret -n demo-namespace  --from-literal foo=bar
    secret/other-secret created
    

    step-2: Create a service account(my-custom-sa) in the default namespace.

    kubectl create sa my-custom-sa
    

    step-3: Validate that, by default, the service account you created in the last step has no access to the secrets present in demo-namespace.

    kubectl auth can-i get secret -n demo-namespace --as system:serviceaccount:default:my-custom-sa
    no
    

    step-4: Create a cluster role with permissions of get and list secrets from demo-namespace namespace.

    kubectl create clusterrole role-for-other-user --verb get,list --resource secret
    clusterrole.rbac.authorization.k8s.io/role-for-other-user created
    

    step-5: Create a rolebinding to bind the cluster role created in last step.

     kubectl create  rolebinding role-for-other-user -n demo-namespace --serviceaccount default:my-custom-sa --clusterrole  role-for-other-user
     rolebinding.rbac.authorization.k8s.io/role-for-other-user created
    

    step-6: validate that the service account in the default ns now has access to the secrets of demo-namespace. (note the difference from step 3)

    kubectl auth can-i get secret -n demo-namespace --as system:serviceaccount:default:my-custom-sa
    yes
    

    step-7: create a pod in default namsepace and mount the service account you created earlier.

    apiVersion: v1
    kind: Pod
    metadata:
      creationTimestamp: null
      labels:
        run: my-pod
      name: my-pod
    spec:
      serviceAccountName: my-custom-sa
      containers:
      - command:
        - sleep
        - infinity
        image: bitnami/kubectl
        name: my-pod
        resources: {}
      dnsPolicy: ClusterFirst
      restartPolicy: Always
    status: {}
    

    step-7: Validate that you can read the secret of demo-namespace from the pod in the default namespace.

     curl -sSk -H "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)"       https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_PORT_443_TCP_PORT/api/v1/namespaces/demo-namespace/secrets
    {
      "kind": "SecretList",
      "apiVersion": "v1",
      "metadata": {
        "resourceVersion": "668709"
      },
      "items": [
        {
          "metadata": {
            "name": "other-secret",
            "namespace": "demo-namespace",
            "uid": "5b3b9dba-be5d-48cc-ab16-4e0ceb3d1d72",
            "resourceVersion": "662043",
            "creationTimestamp": "2022-08-19T14:51:15Z",
            "managedFields": [
              {
                "manager": "kubectl-create",
                "operation": "Update",
                "apiVersion": "v1",
                "time": "2022-08-19T14:51:15Z",
                "fieldsType": "FieldsV1",
                "fieldsV1": {
                  "f:data": {
                    ".": {},
                    "f:foo": {}
                  },
                  "f:type": {}
                }
              }
            ]
          },
          "data": {
            "foo": "YmFy"
          },
          "type": "Opaque"
        }
      ]
    }