Search code examples
spring-bootkubernetesminikubespring-boot-actuator

Spring boot readiness probe fails on kubernetes


I have a Spring Boot app version 3.2.1, that I'm running on Minikube inside WSL, using the Docker driver. The app is really minimal and boots within just a few seconds. When running on localhost, the readiness probe is accessible on http://localhost:8080/actuator/health/readiness and it returns a valid status.

However, when deploying the service & deployment to minikube, I notice that "describe pod" contains:

Liveness probe failed: Get "http://10.244.0.28:8080/actuator/health/liveness": 
dial tcp 10.244.0.28:8080: connect: connection refused 

This causes many consecutive pod restarts, and only after a few minutes the restarts stop and the pods stabilize. The pod logs show no errors, only the standard output that you see when spring boot starts. These are my manifests:

apiVersion: v1
kind: Service
metadata:
  name: my-portal-gateway
  labels:
    helm.sh/chart: my-portal-gateway-0.1.0
    app.kubernetes.io/name: my-portal-gateway
    app.kubernetes.io/instance: my-portal-gateway
    app.kubernetes.io/version: "1.0.0"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  selector:
    app.kubernetes.io/name: my-portal-gateway
    app.kubernetes.io/instance: my-portal-gateway
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 8080
---
# Source: my-portal-gateway/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-portal-gateway
  labels:
    helm.sh/chart: my-portal-gateway-0.1.0
    app.kubernetes.io/name: my-portal-gateway
    app.kubernetes.io/instance: my-portal-gateway
    app.kubernetes.io/version: "1.0.0"
    app.kubernetes.io/managed-by: Helm
spec:
  replicas: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: my-portal-gateway
      app.kubernetes.io/instance: my-portal-gateway
  template:
    metadata:
      labels:
        helm.sh/chart: my-portal-gateway-0.1.0
        app.kubernetes.io/name: my-portal-gateway
        app.kubernetes.io/instance: my-portal-gateway
        app.kubernetes.io/version: "1.0.0"
        app.kubernetes.io/managed-by: Helm
    spec:
       containers:
        - name: my-portal-gateway
          image: "myrepo/my-portal-gateway:1.0.0"
          imagePullPolicy: IfNotPresent
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          livenessProbe:
            httpGet:
              path: /actuator/health/liveness
              port: 8080
          readinessProbe:
            httpGet:
              path: /actuator/health/readiness
              port: 8080
            initialDelaySeconds: 60
            periodSeconds: 20
            failureThreshold: 3
            successThreshold: 1
            timeoutSeconds: 5

My application.yaml file in Spring Boot:

server:
  address: 0.0.0.0

management:
  endpoints:
    web:
      exposure:
        include:
          - health
          - info
  endpoint:
    health:
      group:
        readiness:
          include: readinessProbe
      probes:
        enabled: true

How to solve the pod restarts issue? Thanks.


Solution

  • I finally found a solution - adding an initialDelaySeconds to the liveness probe, in addition to the existing initialDelaySeconds in the readiness probe.