Search code examples
kuberneteskubernetes-helmamazon-eksaerospikecert-manager

Aerospike Kubernetes operator installing error in EKS cluster


strong textTill the cert-manager every pods working good as followed by aerospike docs. But while installing the operator the operator pods get crash loop backoff.

Installing operator using:

git clone https://github.com/aerospike/aerospike-kubernetes-operator.git
git checkout 2.5.0
cd aerospike-kubernetes-operator/helm-charts
helm install aerospike-kubernetes-operator ./aerospike-kubernetes-operator --set replicas=3

Pods running:

PS C:\Users\B.Jimmy\aerospike-kubernetes-operator-1.0.0> kubectl get pods -A
NAMESPACE      NAME                                             READY   STATUS             RESTARTS         AGE
cert-manager   cert-manager-576c79cb45-xkr88                    1/1     Running            0                4h41m
cert-manager   cert-manager-cainjector-664f76bc59-4b5kz         1/1     Running            0                4h41m
cert-manager   cert-manager-webhook-5d4fd5cb7f-f96qx            1/1     Running            0                4h41m
default        aerospike-kubernetes-operator-7bbb8745c8-86884   1/2     CrashLoopBackOff   36 (59s ago)     159m
default        aerospike-kubernetes-operator-7bbb8745c8-jzkww   1/2     Error              36 (5m14s ago)   159m
kube-system    aws-node-7b4nb                                   1/1     Running            0                21h
kube-system    aws-node-llnzh                                   1/1     Running            0                21h
kube-system    coredns-6c97f4f789-fhnq6                         1/1     Running            0                21h
kube-system    coredns-6c97f4f789-wmcdm                         1/1     Running            0                21h
kube-system    kube-proxy-5gwld                                 1/1     Running            0                21h
kube-system    kube-proxy-z2nwk                                 1/1     Running            0                21h
olm            catalog-operator-56db4cd676-hln6h                1/1     Running            0                21h
olm            olm-operator-5b8f867598-7h9z6                    1/1     Running            0                21h
olm            operatorhubio-catalog-bd8rq                      1/1     Running            0                178m
olm            packageserver-7cbbc9c85f-jms5f                   1/1     Running            0                21h
olm            packageserver-7cbbc9c85f-z45jg                   1/1     Running            0                21h

Crashing Pod Log:

PS C:\Users\B.Jimmy\aerospike-kubernetes-operator-1.0.0> kubectl logs -f aerospike-kubernetes-operator-7bbb8745c8-86884
Defaulted container "manager" out of: manager, kube-rbac-proxy
flag provided but not defined: -config
Usage of /manager:
  -health-probe-bind-address string
        The address the probe endpoint binds to. (default ":8081")
  -kubeconfig string
        Paths to a kubeconfig. Only required if out-of-cluster.
  -leader-elect
        Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager.
  -metrics-bind-address string
        The address the metric endpoint binds to. (default ":8080")
  -zap-devel
        Development Mode defaults(encoder=consoleEncoder,logLevel=Debug,stackTraceLevel=Warn). Production Mode defaults(encoder=jsonEncoder,logLevel=Info,stackTraceLevel=Error) (default true)
  -zap-encoder value
        Zap log encoding (one of 'json' or 'console')
  -zap-log-level value
        Zap Level to configure the verbosity of logging. Can be one of 'debug', 'info', 'error', or any integer value > 0 which corresponds to custom debug levels of increasing verbosity
  -zap-stacktrace-level value
        Zap Level at and above which stacktraces are captured (one of 'info', 'error', 'panic').

Do I need to configure nginx ingress after installing cert-manager.


Solution

  • I can recreate a similar behavior by following the steps you provided. I think there may be an accidental mistype with those steps in regards to the branch checkout so its attempting to use the master branch instead of 2.5.0

    The steps should be:

    git clone https://github.com/aerospike/aerospike-kubernetes-operator.git
    cd aerospike-kubernetes-operator/helm-charts
    git checkout 2.5.0
    helm install aerospike-kubernetes-operator ./aerospike-kubernetes-operator --set replicas=3
    

    notice the cd and git checkout commands flipped

    **NOTE: You may need to uninstall the current helm chart first before reinstalling **

    Example:

    helm uninstall aerospike-kubernetes-operator
    

    As a side note: I see you also have OLM namespaces already and may benefit using the OLM installation for AKO found here: https://docs.aerospike.com/cloud/kubernetes/operator/install-operator-operatorhub

    Hopefully this helps!