Search code examples
kubernetesnfs

Create and mount subdirectories in exported NFS directory in Kubernetes

I have a fileserver with a folder /mnt/kubernetes-volumes/. Inside this folder, I have subdirectories for every user of my application:

  • /mnt/kubernetes-volumes/customerA
  • /mnt/kubernetes-volumes/customerB
  • /mnt/kubernetes-volumes/customerC

Inside each customer folder, I have two folders for the applications of my data stores:

  • /mnt/kubernetes-volumes/customerA/elastic
  • /mnt/kubernetes-volumes/customerA/postgres

I have a Kubernetes configuration file that specifies persistent volumes and claims. These mount to the elastic and postgres folders.

# Create a volume on the NFS disk that the postgres db can use. 
apiVersion: v1
kind: PersistentVolume
metadata:
  namespace: customerA
  name: postgres-pv
  labels:
    name: postgres-volume 
spec:
  storageClassName: manual 
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  nfs:
    server: 1.2.3.4 # ip addres of nfs server
    path: "/mnt/kubernetes-volumes/customerA/postgres" 

---

# And another persistent volume for Elastic

Currently, my /etc/exports file looks like this:

/mnt/kubernetes-volumes/customerA/postgres 10.0.0.1(rw,sync,no_subtree_check,no_root_squash)
/mnt/kubernetes-volumes/customerA/elastic 10.0.0.1(rw,sync,no_subtree_check,no_root_squash)
/mnt/kubernetes-volumes/customerA/postgres 10.0.0.2(rw,sync,no_subtree_check,no_root_squash)
/mnt/kubernetes-volumes/customerA/elastic 10.0.0.2(rw,sync,no_subtree_check,no_root_squash)


/mnt/kubernetes-volumes/customerB/postgres 10.0.0.1(rw,sync,no_subtree_check,no_root_squash)
/mnt/kubernetes-volumes/customerB/elastic 10.0.0.1(rw,sync,no_subtree_check,no_root_squash)
/mnt/kubernetes-volumes/customerB/postgres 10.0.0.2(rw,sync,no_subtree_check,no_root_squash)
/mnt/kubernetes-volumes/customerB/elastic 10.0.0.2(rw,sync,no_subtree_check,no_root_squash)

For every customer, I explicitly export a postgres and elastic folder separately for every node in my Kubernetes cluster. This works as intended. However, I now have to manually add rows to the etc/exports file for every new customer. Is it possible to have just two lines like this:

/mnt/kubernetes-volumes/ 10.0.0.1(rw,sync,no_subtree_check,no_root_squash)
/mnt/kubernetes-volumes/ 10.0.0.2(rw,sync,no_subtree_check,no_root_squash)

And automatically have Kubernetes create the correct sub-directories (customer, postgres and elastic) inside the kubernetes-volumes directory and mount them?

Solution

  • Kubernetes can't perform system commands on nodes by itself, you need to use some external tool, such as bash script.

    You can use only the path /mnt/kubernetes-volumes/ in you containers and using environment variable you can pass the customer name, but it will make all data acessible by all other pods, and isn't a good idea.

    Also, you could try to use HELM templates to create your persistentVolumes using the name of your customers as variables.