Search code examples
apache-camelquarkusamazon-eks

Quarkus native application on AWS EKS - "exec format error"

I am trying to deploy a Quarkus-Camel native app to AWS EKS but getting the log error: exec ./application: exec format error

The app works fine in my local minikube environment and I've ensure that the os/arch of the image is linux/arm64. I have another app which I am building & deploying the exact same way which works without any issues.

The pod state (kubectl describe pod) is:

Name:             integration-order-6dbccccb7d-qm9sb
Namespace:        development
Priority:         0
Service Account:  integration-order
Node:             ip-192-168-31-231.ap-southeast-2.compute.internal/192.168.31.231
Start Time:       Sun, 12 Mar 2023 21:49:05 +1100
Labels:           app.kubernetes.io/name=integration-order
                  app.kubernetes.io/version=1.0.0
                  pod-template-hash=6dbccccb7d
Annotations:      app.quarkus.io/build-timestamp: 2023-03-12 - 09:59:34 +0000
                  app.quarkus.io/commit-id: b888f2e29a05851b20d400386d70f3b269c3ed7a
Status:           Running
IP:               192.168.6.84
IPs:
  IP:           192.168.6.84
Controlled By:  ReplicaSet/integration-order-6dbccccb7d
Containers:
  integration-order:
    Container ID:   containerd://6cd1d5c2459d00ea17e28528a06e73846cdad78cb2f4d03696427f88f2107bd2
    Image:          fluentintegrations/integration-order:1.0.0
    Image ID:       docker.io/fluentintegrations/integration-order@sha256:eb42f2faf67cd61b0eb8e805aa6a46d5649db1b4aafd448319cf50bec6f476f2
    Port:           8087/TCP
    Host Port:      0/TCP
    State:          Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Sun, 12 Mar 2023 21:49:22 +1100
      Finished:     Sun, 12 Mar 2023 21:49:22 +1100
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Sun, 12 Mar 2023 21:49:06 +1100
      Finished:     Sun, 12 Mar 2023 21:49:06 +1100
    Ready:          False
    Restart Count:  2
    Liveness:       http-get http://:8087/q/health/live delay=30s timeout=10s period=30s #success=1 #failure=3
    Readiness:      http-get http://:8087/q/health/ready delay=45s timeout=10s period=45s #success=1 #failure=3
    Environment:
      KUBERNETES_NAMESPACE:  development (v1:metadata.namespace)
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-rmczh (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  kube-api-access-rmczh:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age               From               Message
  ----     ------     ----              ----               -------
  Normal   Scheduled  20s               default-scheduler  Successfully assigned development/integration-order-6dbccccb7d-qm9sb to ip-192-168-31-231.ap-southeast-2.compute.internal
  Normal   Pulled     3s (x3 over 20s)  kubelet            Container image "fluentintegrations/integration-order:1.0.0" already present on machine
  Normal   Created    3s (x3 over 20s)  kubelet            Created container integration-order
  Normal   Started    3s (x3 over 20s)  kubelet            Started container integration-order
  Warning  BackOff    2s (x3 over 18s)  kubelet            Back-off restarting failed container

Maven command:

 mvn clean package -Pnative -e \
-Dmaven.test.skip=true \
-Dquarkus.native.remote-container-build=true \
-Dquarkus.kubernetes.deploy=true \
-Dquarkus.container-image.push=true

application.properties:

quarkus.http.port=8087

# quarkus kubernetes extension
quarkus.container-image.group=fluentintegrations
quarkus.container-image.name=integration-order
quarkus.kubernetes.namespace=development
quarkus.kubernetes.readiness-probe.initial-delay=45s
quarkus.kubernetes.readiness-probe.period=45s
quarkus.kubernetes.liveness-probe.initial-delay=30s
quarkus.kubernetes.liveness-probe.period=30s
quarkus.kubernetes.image-pull-policy=ifNotPresent
quarkus.kubernetes.service-type=LoadBalancer

I am running this on a Mac M1. Any help or suggestions on what could go wrong / how to investigate further would be much appreciated.

I've checked & compared most of the configurations but am unable to identify why one works the other doesn't. I have tried all the suggested steps here:https://aws.amazon.com/premiumsupport/knowledge-center/eks-pod-status-troubleshooting/

Solution

  • Thank you @Jeremy Cowan & @gumelaragum!

    Creating the cluster with the parameter --node-type=m6g.large resolved my issue and I'm able to run the images on EKS.

    Question: Is there a way to build the artifacts for a x86_64 architecture on a Mac M1? I found a few threads stating that -Dquarkus.native.container-build=true or using a separate builder image (Issue compiling quarkus native image on Apple Sillicon)resolved the issue, but none of them worked for me and the file type would always state ARM aarch64.

    Would Maven play a role here?