Search code examples
kubernetesdynamicpersistent-volumes

Dynamic storages in Kubernetes

I have an application running on Kubernetes that needs to access SMB shares that are configured dynamically (host, credentials, etc) within said application. I am struggling to achieve this (cleanly) with Kubernetes.

I am facing several difficulties:

  • I do not want "a" storage, I want explicitly specified SMB shares
  • These shares are dynamically defined within the application and not known beforehand
  • I have a variable amount of shares and a single pod needs to be able to access all of them

We currently have a solution where, on each kubernetes worker node, all shares are mounted to mountpoints in a common folder. This folder is then given as HostPath volume to the containers that need access to those storages. Finally, each of those containers has a logic to access the subfolder(s) matching the storage(s) he needs.

The downside, and the reason why I'm looking for a cleaner alternative, is:

  • HostPath volumes present security risks
  • For this solution, I need something outside Kubernetes that mounts the SMB shares automatically on each Kubernetes node

Is there a better solution that I am missing?

The Kubernetes object that seems to match this approach the most closely is the Projected Volume, since it "maps existing volume sources into the same directory". However, it doesn't support the type of volume source I need and I don't think it is possible to add/remove volume sources dynamically without restarting the pods that use this Projected Volume.

Solution

  • For sure your current solution using HostPath on the nodes is not flexible, not secure thus it is not a good practice.

    I think you should consider using one of the custom drivers for your SMB shares:

    CIFS FlexVolume Plugin:

    This solution is older and it is replaced by a CSI Driver. The advantage compared to CSI is that you can specify SMB shares directly from the pod definition (including credentials as Kubernetes secret) as you prefer.

    Here you can find instructions on how to install this plugin on your cluster.

    SMB CSI Driver:

    This driver will automatically take care of mounting SMB shares on all nodes by using DaemonSet.

    You can install SMB CSI Driver either by bash script or by using a helm chart.

    Assuming you have your SMB server ready, you can use one of the following solution to access it from your pod:

    In both cases you have to use a previously created secret with the credentials.

    In your case, for every SMB share you should create a Storage class / PV and mount it to the pod.

    The advantage of CSI Driver is that it is newer, currently maintained solution and it replaced FlexVolume.

    Below is diagram representing how CSI plugin operates:

    Also check: