why elasticsearch cluster in kubernetes start so slow

Question

I am start elasticsearch cluster(v6.4.3) pod in kubernetes cluster(v1.15.2),but the each pod start spent more than 20min. The image file already pull to local,what should I do to make it start faster?

everytime I restart the cluster takes more than 1 hour.Is it possible to make it start it less than 1min?This is my yaml of elasticsearch:

{
  "kind": "StatefulSet",
  "apiVersion": "apps/v1beta2",
  "metadata": {
    "name": "es-cluster",
    "namespace": "logging",
    "selfLink": "/apis/apps/v1beta2/namespaces/logging/statefulsets/es-cluster",
    "uid": "06afbdc1-3ada-401e-8e62-74c6ce4aba97",
    "resourceVersion": "18109360",
    "generation": 44,
    "creationTimestamp": "2020-02-28T06:28:48Z",
    "labels": {
      "app": "elasticsearch"
    },
    "annotations": {
      "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"apps/v1\",\"kind\":\"StatefulSet\",\"metadata\":{\"annotations\":{},\"name\":\"es-cluster\",\"namespace\":\"logging\"},\"spec\":{\"replicas\":1,\"selector\":{\"matchLabels\":{\"app\":\"elasticsearch\"}},\"serviceName\":\"elasticsearch\",\"template\":{\"metadata\":{\"labels\":{\"app\":\"elasticsearch\"}},\"spec\":{\"containers\":[{\"env\":[{\"name\":\"cluster.name\",\"value\":\"k8s-logs\"},{\"name\":\"node.name\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.name\"}}},{\"name\":\"discovery.zen.ping.unicast.hosts\",\"value\":\"es-cluster-0.elasticsearch,es-cluster-1.elasticsearch,es-cluster-2.elasticsearch\"},{\"name\":\"discovery.zen.minimum_master_nodes\",\"value\":\"2\"},{\"name\":\"ES_JAVA_OPTS\",\"value\":\"-Xms512m -Xmx512m\"}],\"image\":\"registry.cn-hangzhou.aliyuncs.com/dabai_app_k8s/dabai_fat/elasticsearch-oss:6.4.3\",\"name\":\"elasticsearch\",\"ports\":[{\"containerPort\":9200,\"name\":\"rest\",\"protocol\":\"TCP\"},{\"containerPort\":9300,\"name\":\"inter-node\",\"protocol\":\"TCP\"}],\"resources\":{\"limits\":{\"cpu\":\"1000m\"},\"requests\":{\"cpu\":\"100m\"}},\"volumeMounts\":[{\"mountPath\":\"/usr/share/elasticsearch/data\",\"name\":\"data\"}]}],\"imagePullSecrets\":[{\"name\":\"regcred\"}],\"initContainers\":[{\"command\":[\"sh\",\"-c\",\"chown -R 1000:1000 /usr/share/elasticsearch/data\"],\"image\":\"docker.io/library/busybox:1.31.1\",\"name\":\"fix-permissions\",\"securityContext\":{\"privileged\":true},\"volumeMounts\":[{\"mountPath\":\"/usr/share/elasticsearch/data\",\"name\":\"data\"}]},{\"command\":[\"sysctl\",\"-w\",\"vm.max_map_count=262144\"],\"image\":\"docker.io/library/busybox:1.31.1\",\"name\":\"increase-vm-max-map\",\"securityContext\":{\"privileged\":true}},{\"command\":[\"sh\",\"-c\",\"ulimit -n 65536\"],\"image\":\"docker.io/library/busybox:1.31.1\",\"name\":\"increase-fd-ulimit\",\"securityContext\":{\"privileged\":true}}]}},\"volumeClaimTemplates\":[{\"metadata\":{\"labels\":{\"app\":\"elasticsearch\"},\"name\":\"data\"},\"spec\":{\"accessModes\":[\"ReadWriteOnce\"],\"resources\":{\"requests\":{\"storage\":\"5Gi\"}},\"storageClassName\":\"es-data-db\"}}]}}\n"
    }
  },
  "spec": {
    "replicas": 3,
    "selector": {
      "matchLabels": {
        "app": "elasticsearch"
      }
    },
    "template": {
      "metadata": {
        "creationTimestamp": null,
        "labels": {
          "app": "elasticsearch"
        }
      },
      "spec": {
        "initContainers": [
          {
            "name": "fix-permissions",
            "image": "docker.io/library/busybox:1.31.1",
            "command": [
              "sh",
              "-c",
              "chown -R 1000:1000 /usr/share/elasticsearch/data"
            ],
            "resources": {},
            "volumeMounts": [
              {
                "name": "data",
                "mountPath": "/usr/share/elasticsearch/data"
              }
            ],
            "terminationMessagePath": "/dev/termination-log",
            "terminationMessagePolicy": "File",
            "imagePullPolicy": "IfNotPresent",
            "securityContext": {
              "privileged": true
            }
          },
          {
            "name": "increase-vm-max-map",
            "image": "docker.io/library/busybox:1.31.1",
            "command": [
              "sysctl",
              "-w",
              "vm.max_map_count=262144"
            ],
            "resources": {},
            "terminationMessagePath": "/dev/termination-log",
            "terminationMessagePolicy": "File",
            "imagePullPolicy": "IfNotPresent",
            "securityContext": {
              "privileged": true
            }
          },
          {
            "name": "increase-fd-ulimit",
            "image": "docker.io/library/busybox:1.31.1",
            "command": [
              "sh",
              "-c",
              "ulimit -n 65536"
            ],
            "resources": {},
            "terminationMessagePath": "/dev/termination-log",
            "terminationMessagePolicy": "File",
            "imagePullPolicy": "IfNotPresent",
            "securityContext": {
              "privileged": true
            }
          }
        ],
        "containers": [
          {
            "name": "elasticsearch",
            "image": "registry.cn-hangzhou.aliyuncs.com/dabai_app_k8s/dabai_fat/elasticsearch:6.4.3",
            "ports": [
              {
                "name": "rest",
                "containerPort": 9200,
                "protocol": "TCP"
              },
              {
                "name": "inter-node",
                "containerPort": 9300,
                "protocol": "TCP"
              }
            ],
            "env": [
              {
                "name": "cluster.name",
                "value": "k8s-logs"
              },
              {
                "name": "node.name",
                "valueFrom": {
                  "fieldRef": {
                    "apiVersion": "v1",
                    "fieldPath": "metadata.name"
                  }
                }
              },
              {
                "name": "discovery.zen.ping.unicast.hosts",
                "value": "es-cluster-0.elasticsearch,es-cluster-1.elasticsearch,es-cluster-2.elasticsearch"
              },
              {
                "name": "discovery.zen.minimum_master_nodes",
                "value": "2"
              },
              {
                "name": "ES_JAVA_OPTS",
                "value": "-Xms2048m -Xmx2048m"
              }
            ],
            "resources": {
              "limits": {
                "cpu": "1"
              },
              "requests": {
                "cpu": "100m"
              }
            },
            "volumeMounts": [
              {
                "name": "data",
                "mountPath": "/usr/share/elasticsearch/data"
              }
            ],
            "terminationMessagePath": "/dev/termination-log",
            "terminationMessagePolicy": "File",
            "imagePullPolicy": "IfNotPresent"
          }
        ],
        "restartPolicy": "Always",
        "terminationGracePeriodSeconds": 30,
        "dnsPolicy": "ClusterFirst",
        "nodeSelector": {
          "app-type": "assistant-app"
        },
        "securityContext": {},
        "imagePullSecrets": [
          {
            "name": "regcred"
          }
        ],
        "schedulerName": "default-scheduler"
      }
    },
    "volumeClaimTemplates": [
      {
        "metadata": {
          "name": "data",
          "creationTimestamp": null,
          "labels": {
            "app": "elasticsearch"
          }
        },
        "spec": {
          "accessModes": [
            "ReadWriteOnce"
          ],
          "resources": {
            "requests": {
              "storage": "5Gi"
            }
          },
          "storageClassName": "es-data-db",
          "volumeMode": "Filesystem"
        },
        "status": {
          "phase": "Pending"
        }
      }
    ],
    "serviceName": "elasticsearch",
    "podManagementPolicy": "OrderedReady",
    "updateStrategy": {
      "type": "RollingUpdate",
      "rollingUpdate": {
        "partition": 0
      }
    },
    "revisionHistoryLimit": 10
  },
  "status": {
    "observedGeneration": 44,
    "replicas": 2,
    "readyReplicas": 1,
    "currentReplicas": 2,
    "updatedReplicas": 2,
    "currentRevision": "es-cluster-6b6647d94",
    "updateRevision": "es-cluster-6b6647d94",
    "collisionCount": 0
  }
}

Answer 1

Because the data is not important,I delete the cluster data folder and restart.It is fast.Read the elasticsearch file and remove the init container chown command should fix this.chown when first time run your clusterif needed directory permission.Just remove:

         {
            "name": "fix-permissions",
            "image": "docker.io/library/busybox:1.31.1",
            "command": [
              "sh",
              "-c",
              "chown -R 1000:1000 /usr/share/elasticsearch/data"
            ],
            "resources": {},
            "volumeMounts": [
              {
                "name": "data",
                "mountPath": "/usr/share/elasticsearch/data"
              }
            ],
            "terminationMessagePath": "/dev/termination-log",
            "terminationMessagePolicy": "File",
            "imagePullPolicy": "IfNotPresent",
            "securityContext": {
              "privileged": true
            }
          }