Search code examples
elasticsearchkubernetes

Why does it complain about `volume claim templates can only have their storage requests increased`?


I am deploying Elasticsearch cluster on Kubernetes in AWS EKS. The spec I have is:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: es
spec:
  version: 7.14.0
  nodeSets:
  - name: node
    count: 2
    config:
      node.store.allow_mmap: false
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
          - ReadWriteOnce
        storageClassName: ebs-sc
        resources:
          requests:
            storage: 1024Gi

When I deploy I got this error:

for: "es.yml": admission webhook "elastic-es-validation-v1.k8s.elastic.co" denied the request: Elasticsearch.elasticsearch.k8s.elastic.co "es" is invalid: spec.nodeSet[0].volumeClaimTemplates: Invalid value: []v1.PersistentVolumeClaim{v1.PersistentVolumeClaim{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"elasticsearch-data", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PersistentVolumeClaimSpec{AccessModes:[]v1.PersistentVolumeAccessMode{"ReadWriteOnce"}, Selector:(*v1.LabelSelector)(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"storage":resource.Quantity{i:resource.int64Amount{value:1099511627776, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"", Format:"BinarySI"}}}, VolumeName:"", StorageClassName:(*string)(0xc000cd2600), VolumeMode:(*v1.PersistentVolumeMode)(nil), DataSource:(*v1.TypedLocalObjectReference)(nil)}, Status:v1.PersistentVolumeClaimStatus{Phase:"", AccessModes:[]v1.PersistentVolumeAccessMode(nil), Capacity:v1.ResourceList(nil), Conditions:[]v1.PersistentVolumeClaimCondition(nil)}}}: volume claim templates can only have their storage requests increased, if the storage class allows volume expansion. Any other change is forbidden

The spec includes volumeClaimTemplates which is used to claim the persistent storage. I don't understand why it says volume claim templates can only have their storage requests increased, if the storage class allows volume expansion.

I have checked PVC is empty:

$ kubectl get pvc
No resources found in default namespace.

And I have below spec for storage class:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: ebs-sc
provisioner: ebs.csi.aws.com
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer

Solution

  • ...I don't understand why it says volume claim templates can only have their storage requests increased, if the storage class allows volume expansion.

    With ECK, when you applied a spec that referred a StorageClass that allows volume expansion, you can increase the storage size and re-apply the spec. Beside this no other change is allowed for the volumeClaimTemplates section. If you must change the volumeClaimTemplates (eg. refer a different StorageClass, reduce storage size), you need to also rename the name under nodeSets and re-apply the spec. Example:

    ...
    spec:
      ...
      nodeSets:
      - name: <CHANGEME!>  # <-- MUST change if not a simple size increase
        ...
        volumeClaimTemplates:
        - metadata:
            name: elasticsearch-data
          spec:
            resources:
              requests:
                storage: <xxxGi>  # <-- Can be increased if StorageClass supports expansion. To decrease the nodeSets name MUST be changed.
            storageClassName:<changeable only if nodeSets name is changed>