Search code examples
kuberneteskubernetes-pod

Is PSP only for pods created through deplyment/replica set?


I am trying to set up the security polices in the cluster. I enabled pod security and created a restricted psp

1.Step 1 - Created PSP 2.Step 2 - Created Cluster Role 3.Step 3 - Create ClusterRoleBinding

PSP

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  annotations:
    serviceaccount.cluster.cattle.io/pod-security: restricted
    serviceaccount.cluster.cattle.io/pod-security-version: "2315292"
  creationTimestamp: "2022-02-28T20:48:12Z"
  labels:
    cattle.io/creator: norman
  name: restricted-psp
spec:
  allowPrivilegeEscalation: false
  fsGroup:
    ranges:
    - max: 65535
      min: 1
    rule: MustRunAs
  requiredDropCapabilities:
  - ALL
  runAsUser:
    rule: RunAsAny
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    ranges:
    - max: 65535
      min: 1
    rule: MustRunAs
  volumes:
  - configMap
  - emptyDir
  - projected
  - secret
  - downwardAPI
  - persistentVolumeClaim

Cluster Role -

apiVersion: rbac.authorization.k8s.io/v1
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    serviceaccount.cluster.cattle.io/pod-security: restricted
  labels:
    cattle.io/creator: norman
  name: restricted-clusterrole
rules:
- apiGroups:
  - extensions
  resourceNames:
  - restricted-psp
  resources:
  - podsecuritypolicies
  verbs:
  - use

ClusterRoleBinding

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: restricted-crb
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: restricted-clusterrole
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts:security
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:authenticated

Create couple of yams one for deplyment and other for pod

kubectl create ns security

$ cat previleged-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  labels:
    app: privileged-deploy
  name: privileged-pod
spec:
      containers:
        - image: alpine
          name: alpine
          stdin: true
          tty: true
          securityContext:
            privileged: true
      hostPID: true
      hostNetwork: true

$ cat previleged-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: privileged-deploy
  name: privileged-deploy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: privileged-deploy
  template:
    metadata:
      labels:
        app: privileged-deploy
    spec:
      containers:
        - image: alpine
          name: alpine
          stdin: true
          tty: true
          securityContext:
            privileged: true
      hostPID: true
      hostNetwork: true

The expectation was both pod and deployment to be prevented . But the pod got created and deployment failed

$ kg all -n security
NAME                 READY   STATUS    RESTARTS   AGE
**pod/privileged-pod   1/1     Running   0          13m**

NAME                                READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/privileged-deploy   0/1     0            0           13m

NAME                                           DESIRED   CURRENT   READY   AGE
replicaset.apps/privileged-deploy-77d7c75dd8   1         0         0       13m

As Expected Error for Deployment came as below

Events:
  Type     Reason        Age                   From                   Message
  ----     ------        ----                  ----                   -------
  Warning  FailedCreate  3m10s (x18 over 14m)  replicaset-controller  Error creating: pods "privileged-deploy-77d7c75dd8-" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used spec.securityContext.hostPID: Invalid value: true: Host PID is not allowed to be used spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used spec.securityContext.hostPID: Invalid value: true: Host PID is not allowed to be used spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used spec.securityContext.hostPID: Invalid value: true: Host PID is not allowed to be used spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]

But the pod created directly though yaml worked . Is PSP only for pods getting created through deplyment/rs ? Please help , how can we prevent users from creating pods which are previleged and dangerous


Solution

  • But the pod created directly though yaml worked . Is PSP only for pods getting created through deplyment/rs ?

    That's because when you create a bare pod (creating a pod directly) it will be created by the user called kubernetes-admin (in default scenarios), who is a member of the group system:masters, which is mapped to a cluster role called cluster-admin, which has access to all the PSPs that get created on the cluster. So the creation of bare pods will be successful.

    Whereas pods that are created by deployment,rs,sts,ds (all the managed pods) will be created using the service account mentioned in their definition. The creation of these pods will be successful only if these service accounts have access to PSP via a cluster role or role.

    how can we prevent users from creating pods which are previleged and dangerous

    We need to identify what is that user and group that will be creating these pods (by checking ~/kube/config or its certificate) and then make sure, it does not have access to PSP via any cluster role or role.