I have 2 apps. One is config-server and other is business-logic-app that consumes data from config-server. Both are running on same namespace in Kubernetes (kubectl on my laptop). However, am getting connection timed out exception when business-logic-app is connecting to config-server which is leading to livenessProbe and readinessProbe failures. What am I missing?
config-server.yaml
# Config server
apiVersion: apps/v1
kind: Deployment
metadata:
name: kubernetes-learning-config-server
namespace: kubernetes-learning
labels:
app: kubernetes-learning-config-server
spec:
replicas: 2
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
selector:
matchLabels:
app: kubernetes-learning-config-server
template:
metadata:
name: kubernetes-learning-config-server
labels:
app: kubernetes-learning-config-server
spec:
containers:
- name: kubernetes-learning-config-server
image: ghcr.io/kubernetes/learning.config-server
imagePullPolicy: Always
ports:
- containerPort: 8888
protocol: TCP
- containerPort: 48888
protocol: TCP
env:
- name: BPL_JVM_THREAD_COUNT
value: "50"
- name: BPL_DEBUG_ENABLED
value: "true"
- name: BPL_DEBUG_PORT
value: "48888"
- name: GITHUB_CONFIG_DATA_URL
value: https://github.com/kubernetes/config-data
- name: GITHUB_CONFIG_DATA_USERNAME
value: github_user
- name: GITHUB_CONFIG_DATA_PERSONAL_ACCESS_TOKEN
value: github_sampletoken
livenessProbe:
httpGet:
path: /alpha-app/local
port: 8888
initialDelaySeconds: 30
periodSeconds: 20
timeoutSeconds: 10
successThreshold: 1
failureThreshold: 3
readinessProbe:
httpGet:
path: /alpha-app/local
port: 8888
initialDelaySeconds: 30
periodSeconds: 20
timeoutSeconds: 10
successThreshold: 1
failureThreshold: 3
restartPolicy: Always
# Expose Config server
---
apiVersion: v1
kind: Service
metadata:
name: kubernetes-learning-config-server
labels:
app: kubernetes-learning-config-server
spec:
type: ClusterIP
selector:
app: kubernetes-learning-config-server
ports:
- protocol: TCP
port: 80
targetPort: 8888
alpha-app.yaml
# app applications
apiVersion: apps/v1
kind: Deployment
metadata:
name: kubernetes-learning-app
namespace: kubernetes-learning
labels:
app: kubernetes-learning-app
spec:
replicas: 1
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
selector:
matchLabels:
app: kubernetes-learning-app
template:
metadata:
name: kubernetes-learning-app
labels:
app: kubernetes-learning-app
spec:
containers:
- name: kubernetes-learning-alpha-app
image: ghcr.io/kubernetes/learning.alpha-app
imagePullPolicy: Always
ports:
- containerPort: 8441
protocol: TCP
- containerPort: 48441
protocol: TCP
env:
- name: BPL_JVM_THREAD_COUNT
value: "50"
- name: BPL_DEBUG_ENABLED
value: "true"
- name: BPL_DEBUG_PORT
value: "48441"
- name: SPRING_PROFILES_ACTIVE
value: kube
- name: SPRING_CLOUD_CONFIG_FAIL_FAST
value: "true"
- name: SPRING_CLOUD_CONFIG_RETRY_INITIAL_INTERVAL
value: "1000"
- name: SPRING_CLOUD_CONFIG_RETRY_MAX_INTERVAL
value: "10000"
- name: SPRING_CLOUD_CONFIG_RETRY_MULTIPLIER
value: "2"
- name: SPRING_CLOUD_CONFIG_RETRY_MAX_ATTEMPTS
value: "5"
- name: SPRING_CLOUD_CONFIG_URI
value: http://kubernetes-learning-config-server:8888
livenessProbe:
httpGet:
path: /info
port: 8441
initialDelaySeconds: 60
timeoutSeconds: 15
periodSeconds: 30
successThreshold: 1
failureThreshold: 3
readinessProbe:
httpGet:
path: /info
port: 8441
initialDelaySeconds: 60
timeoutSeconds: 15
periodSeconds: 30
successThreshold: 1
failureThreshold: 3
restartPolicy: Always
# Expose Config server
---
apiVersion: v1
kind: Service
metadata:
name: kubernetes-learning-app
labels:
app: kubernetes-learning-app
spec:
type: ClusterIP
selector:
app: kubernetes-learning-app
ports:
- protocol: TCP
port: 80
targetPort: 8441
exception
Caused by: org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://kubernetes-learning-config-server:8888/alpha-app/kube": Connect timed out
at org.springframework.web.client.RestTemplate.createResourceAccessException(RestTemplate.java:926) ~[spring-web-6.2.1.jar:6.2.1]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:906) ~[spring-web-6.2.1.jar:6.2.1]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:801) ~[spring-web-6.2.1.jar:6.2.1]
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:683) ~[spring-web-6.2.1.jar:6.2.1]
at org.springframework.cloud.config.client.ConfigServerConfigDataLoader.getRemoteEnvironment(ConfigServerConfigDataLoader.java:349) ~[spring-cloud-config-client-4.2.0.jar:4.2.0]
at org.springframework.cloud.config.client.ConfigServerConfigDataLoader.doLoad(ConfigServerConfigDataLoader.java:130) ~[spring-cloud-config-client-4.2.0.jar:4.2.0]
... 37 common frames omitted
Caused by: java.net.SocketTimeoutException: Connect timed out
at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(Unknown Source) ~[na:na]
at java.base/sun.nio.ch.NioSocketImpl.connect(Unknown Source) ~[na:na]
at java.base/java.net.Socket.connect(Unknown Source) ~[na:na]
at java.base/sun.net.NetworkClient.doConnect(Unknown Source) ~[na:na]
at java.base/sun.net.www.http.HttpClient.openServer(Unknown Source) ~[na:na]
at java.base/sun.net.www.http.HttpClient.openServer(Unknown Source) ~[na:na]
at java.base/sun.net.www.http.HttpClient.<init>(Unknown Source) ~[na:na]
at java.base/sun.net.www.http.HttpClient.New(Unknown Source) ~[na:na]
at java.base/sun.net.www.http.HttpClient.New(Unknown Source) ~[na:na]
at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source) ~[na:na]
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source) ~[na:na]
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source) ~[na:na]
at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source) ~[na:na]
at org.springframework.http.client.SimpleClientHttpRequest.executeInternal(SimpleClientHttpRequest.java:79) ~[spring-web-6.2.1.jar:6.2.1]
at org.springframework.http.client.AbstractStreamingClientHttpRequest.executeInternal(AbstractStreamingClientHttpRequest.java:71) ~[spring-web-6.2.1.jar:6.2.1]
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:81) ~[spring-web-6.2.1.jar:6.2.1]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:900) ~[spring-web-6.2.1.jar:6.2.1]
... 41 common frames omitted
That is because the service is listening on port 80 that will be routed to port 8888 on the target pods.
ports:
- protocol: TCP
port: 80
targetPort: 8888
Therefore, you need to point kubernetes-learning-app to http://kubernetes-learning-config-server:80/alpha-app/kube.
N.B. as mentioned by a comment to the question, http://kubernetes-learning-config-server works too because an http:// URL without port number will default to port 80 which coincidentally what you have set as the service port, not because the service decides the port. Should you use another port like 8080, the URL without a port number would not work.