Search code examples
kubernetesfilesystemsstoragenfs

Solution for user data storage on Google Kubernetes Engine

I'm developing a SaaS solution on GCP, service mainly running GKE. Service will provide each user with a private space on disk (folder to keep their files and folder).

Technical requirements:

  • Mount with ReadWriteMany to multiple pods
  • The number of users: around a few hundred thousand.
  • Each user's private storage size will be pretty small around a few hundred MB each up to a few GB max. But the number of files will be pretty huge, ~ few thousands to hundred thousand
  • File access will be mostly small reads and writes.
  • There will be symbolic links to a shared system folder, but no symbolic link between users' folders.

I currently implement an NFS solution based on this, which is a pain to manage and the project is no longer maintained. I have to manually increase the size of backing PVC and restart all the pods that the PV is mounted in case I need to expand the storage capacity.

Filestore seems to be a very good alternative, but the cost is a big problem as it only allows minimum 1TB storage to be provisioned.

I'm looking for a solution that is easy to manage, good-enough performance and cost-effective. Any recommendation is appreciated.

Thank you very much.

Solution

  • You didn't specify what your end goal is but some alternatives:

    Some of the solutions above will still involve restarting the pods.

    ✌️