Search code examples
dockerkubernetesstoragepersistent-volumeskubernetes-secrets

Mounting a large file in Kubernetes


We are running a pod in Kubernetes that needs to load a file during runtime. This file has the following properties:

  • It is known at build time
  • It should be mounted read-only by multiple pods (the same kind)
  • It might change (externally to the cluster) and needs to be updated
  • For various reasons (security being the main concern) the file cannot be inside the docker image
  • It is potentially quite large, theoretically up to 100 MB, but in practice between 200kB - 10MB.

We have considered various options:

  • Creating a persistent volume, mount the volume in a temporary pod to write (update) the file, unmount the volume, and then mount it in the service with ROX (Read-Only Multiple) claims. This solution means we need downtime during upgrade, and it is hard to automate (due to timings).
  • Creating multiple secrets using the secrets management of Kubernetes, and then "assemble" the file before loading it in an init-container or something similar.

Both of these solutions feels a little bit hacked - is there a better solution out there that we could utilize for solving this?


Solution

  • You need to use a shared filesystem that supports Read/Write Multiple Pods. Here is a link to the CSI Drivers which can be used with Kubernetes and provide those access: https://kubernetes-csi.github.io/docs/drivers.html

    Ideally, you need a solution that is not an appliance, and can run anywhere meaning it can run in the cloud or on-prem.

    The platforms that could work for you are Ceph, GlusterFS, and Quobyte (Disclaimer, I work for Quobyte)