Search code examples
kubernetesamazon-ekskubernetes-pvc

Pod access PVC subdirectory that already existed

I have a pod created using a deployment using git-sync image and mount the volume to a PVC

kind: Deployment
metadata:
  name: config
  namespace: test
spec:
  replicas: 1
  selector:
    matchLabels:
      demo: config
  template:
    metadata:
      labels:
        demo: config
    spec:
      containers:
        - args:
            - '-ssh'
            - '-repo=git@domain.com:org/repo.git'
            - '-dest=conf'
            - '-branch=master'
            - '-depth=1'
          image: 'k8s.gcr.io/git-sync:v3.1.1'
          name: git-sync
          securityContext:
            runAsUser: 65533
          volumeMounts:
            - mountPath: /etc/git-secret
              name: git-secret
              readOnly: true
            - mountPath: /config
              name: cus-config
      securityContext:
        fsGroup: 65533
      volumes:
        - name: git-secret
          secret:
            defaultMode: 256
            secretName: git-creds
        - name: cus-config
          persistentVolumeClaim:
            claimName: cus-config

After the deployment, I checked the pod and got a file path like this.
/tmp/git/conf/subdirA/some.Files
Then I created a second pod from another deployment and want to mount the tmp/git/conf/subdirA on the second pod. This is the example of my second deployment script. 

kind: Deployment
metadata:
  name: test-mount-config
  namespace: test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - image: 'nginx:1.7.9'
          name: nginx
          ports:
            - containerPort: 80
          volumeMounts:
            - mountPath: /root/conf
              name: config
              subPath: tmp/git/conf/subdirA
      volumes:
        - name: config
          persistentVolumeClaim:
            claimName: cus-config

This is my PVC 

kind: PersistentVolumeClaim
metadata:
  annotations:
    volume.beta.kubernetes.io/storage-class: conf
  name: config
  namespace: test
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Mi
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: conf
  namespace: test
provisioner: spdbyz
reclaimPolicy: Retain

I already read about subpath on PVC, but everytime I checked the folder /root/conf on the second pod, there is nothing inside it.
Any idea on how to mount specific PVC subdirectory on another pod?

Solution

  • Very basic example on how share file content between PODs using PV/PVC

    First Create a persistent volume refer below yaml example with hostPath configuration

    apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-pv-1
  labels:
    pv: my-pv-1
spec:
  capacity:
    storage: 1Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: /var/log/mypath

$ kubectl create -f pv.yaml
persistentvolume/my-pv-1 created

    Second create a persistent volume claim using below yaml example 

    apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc-claim-1
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  selector:
    matchLabels:
      pv: my-pv-1


$ kubectl create -f pvc.yaml
persistentvolumeclaim/my-pvc-claim-1 created

    Verify the pv and pvc STATUS is set to BOUND 

    $ kubectl get persistentvolume
NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                  STORAGECLASS   REASON   AGE
my-pv-1   1Gi        RWX            Retain           Bound    default/my-pvc-claim-1                         62s

$ kubectl get persistentvolumeclaims
NAME             STATUS   VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
my-pvc-claim-1   Bound    my-pv-1   1Gi        RWX                           58

    Third consume the pvc in required PODs refer below example yaml where the volume is mounted on two pods nginx-1 and nginx-2.

    apiVersion: v1
kind: Pod
metadata:
  name: nginx-1
spec:
  containers:
  - image: nginx
    name: nginx-1
    volumeMounts:
    - mountPath: /var/log/mypath
      name: test-vol
      subPath: TestSubPath
  volumes:
  - name: test-vol
    persistentVolumeClaim:
        claimName: my-pvc-claim-1


$ kubectl create -f nginx-1.yaml
pod/nginx-1 created


$ kubectl get pods -o wide
NAME             READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
nginx-1   1/1     Running   0          35s   10.244.3.53   k8s-node-3   <none>           <none>

    Create second POD and consume same PVC

    apiVersion: v1
kind: Pod
metadata:
  name: nginx-2
spec:
  containers:
  - image: nginx
    name: nginx-2
    volumeMounts:
    - mountPath: /var/log/mypath
      name: test-vol
      subPath: TestSubPath
  volumes:
  - name: test-vol
    persistentVolumeClaim:
        claimName: my-pvc-claim-1


$ kubectl create -f nginx-2.yaml
pod/nginx-2 created


$ kubectl get pods -o wide
NAME             READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
nginx-1   1/1     Running   0          55s   10.244.3.53   k8s-node-3   <none>           <none>
nginx-2   1/1     Running   0          35s   10.244.3.54   k8s-node-3   <none>           <none>

    Test by connecting to container 1 and write to the file on mount-path.

    root@nginx-1:/# df -kh
Filesystem      Size  Used Avail Use% Mounted on
overlay          12G  7.3G  4.4G  63% /
tmpfs            64M     0   64M   0% /dev
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/vda1        12G  7.3G  4.4G  63% /etc/hosts
shm              64M     0   64M   0% /dev/shm
tmpfs           3.9G   12K  3.9G   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs           3.9G     0  3.9G   0% /proc/acpi
tmpfs           3.9G     0  3.9G   0% /proc/scsi
tmpfs           3.9G     0  3.9G   0% /sys/firmware
root@nginx-1:/# cd /var/log/mypath/
root@nginx-1:/var/log/mypath# date >> date.txt
root@nginx-1:/var/log/mypath# date >> date.txt
root@nginx-1:/var/log/mypath# cat date.txt
Thu Jan 30 10:44:42 UTC 2020
Thu Jan 30 10:44:43 UTC 2020

    Now connect tow second POD/container and it should see the file from first as below

    $ kubectl exec -it nginx-2 -- /bin/bash
root@nginx-2:/# cat /var/log/mypath/date.txt
Thu Jan 30 10:44:42 UTC 2020
Thu Jan 30 10:44:43 UTC 2020