In Kubernetes, can I have a deployment with both read only and writable files?

Question

I am currently running a deployment that is pulling from a database and is set as read only. Unfortunately the deployment freezes on coaction without notice so I came up with the idea of having the code write to a log file and have the liveness probe check to see if the file is up to date.

This has been tested and it works great, however, the issue is getting past the read only part. As you well know, I can not create or write to a file in read only mode so when my code attempts this, I get a permission denied error and the deployment ends. I thought I could fix this by including two file paths in the container and using:

allowedHostPaths:
- pathPrefix: "/code"
  readonly: true
- pathPrefix: "/code/logs"
  readonly: false

and tell the code to wright in the "logs" file, but this did not work. Is there a why to have all my code in a read only file but also have a log file that can be read and written to?

Here is my Dockerfile:

FROM python:3.9.7-alpine3.14
RUN mkdir -p /code/logs
WORKDIR /code
COPY requirements.txt .
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
COPY src/ .
CMD [ "python", "code.py"]

and here is my yaml file:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'docker/default,runtime/default'
    apparmor.security.beta.kubernetes.io/allowedProfileNames: 'runtime/default'
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  'runtime/default'
    apparmor.security.beta.kubernetes.io/defaultProfileName:  'runtime/default'
spec:
  privileged: false # Required to prevent escalations to root.
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
    - ALL
  volumes: # Allow core volume types.
    - 'configMap'
    - 'emptyDir'
    - 'projected'
    - 'secret'
    - 'downwardAPI'
    - 'persistentVolumeClaim'
  hostNetwork: false
  hostIPC: false
  hostPID: false
  runAsUser:
    rule: 'MustRunAsNonRoot'
  seLinux:
    rule: 'RunAsAny'
 supplementalGroups:
    rule: 'MustRunAs'
    ranges:
      - min: 1
        max: 65535
 runAsGroup:
   rule: 'MustRunAs'
   ranges:
     - min: 1
       max: 65535
 fsGroup:
   rule: 'MustRunAs'
   ranges:
     - min: 1
       max: 65535
 readOnlyRootFilesystem: true
 allowedHostPaths:
   - pathPrefix: "/code"
     readOnly: true
   - pathPrefix: "/code/logs"
     readOnly: false

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: psp:restricted 
rules:
  - apiGroups: 
    - policy
    resources: 
      - podsecuritypolicies
    verbs:
      - use
    resourceNames:
      - restricted 

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: manage-app
  namespace: default
subjects:
  - kind: User
    name: my-app-sa
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: psp:restricted
  apiGroup: rbac.authorization.k8s.io

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  selector:
    matchLabels:
      app: orbservice
  template:
    metadata:
      labels:
        app: orbservice
    spec:
      serviceAccountName: my-app-sa
      securityContext:
        runAsUser: 1000
        runAsGroup: 3000
        fsGroup: 2000
      containers:
        - name: my-app
          image: app:v0.0.1
          resources:
            limits:
              memory: "128Mi"
              cpu: "500m"
          livenessProbe:
            exec:
              command:
                - python
                - check_logs.py
            initialDelaySeconds: 60
            periodSeconds: 30
            failureThreshold: 1

An explanation of the solution or errors you see would be most appreciated. Thanks!

Answer 1

allowedHostPaths is related to hostPath volume mounts (aka bind-mounting in folder from the host system). Nothing to do with stuff inside the container image. What you probably want is readOnlyRootFilesystem: true like you have now plus an emptyDir volume mounted at /code/logs set up to be writable by the container's user.