Search code examples
javaspringkubernetesspring-cloudspring-cloud-config

Spring cloud Kubernetes reload timing issue


I stumbled upon a very subtle issue trying to implement ConfigMap property source live-reloading for my app deployed to K8S.

Here are a few config snippets from my current project:

application.yaml

spring:
  application:
    name: myapp
  cloud:
    kubernetes:
      config:
        enabled: true
        name: myapp
        namespace: myapp
        sources:
          - namespace: myapp
            name: myapp-configmap
      reload:
        enabled: true
        mode: event
        strategy: refresh
    refresh:
      refreshable:
        - com.myapp.PropertiesConfig
      extra-refreshable:
        - javax.sql.DataSource

myapp-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    name: myapp
  name: myapp
  namespace: myapp
spec:
  replicas: 1
  selector:
    matchLabels:
      name: myapp-backend
  strategy:
    rollingUpdate:
      maxUnavailable: 0
  template:
    metadata:
      labels:
        name: myapp-backend
    spec:
      serviceAccountName: myapp-config-reader
      volumes:
        - name: myapp-configmap
          configMap:
            name: myapp-configmap
      containers:
        - name: myapp
          image: eu.gcr.io/myproject/myapp:latest
          volumeMounts:
            - name: myapp-configmap
              mountPath: /config
          ports:
            - containerPort: 8080
          envFrom:
            - configMapRef:
                name: myapp-configmap
          env:
            - name: DATASOURCE_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: myapp-db-credentials
                  key: password

myapp-configmap.yml

apiVersion: v1
kind: ConfigMap
metadata:
  name: myapp-configmap
  namespace: myapp
data:
  SPRING_PROFILES_ACTIVE: dtest
  application.yml: |-
     reload.message: 1

PropertiesConfig.java

@Data
@Configuration
@ConfigurationProperties(prefix = "reload")
public class PropertiesConfig {

  private String message;

}

I'm using the following dependencies, in my maven POM:

<parent>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-parent</artifactId>
  <version>2.2.4.RELEASE</version>
</parent>
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-kubernetes-config</artifactId>
  <version>1.1.1.RELEASE</version>
</dependency>

I can successfully deploy myapp to my K8S cluster.

I have a scheduled task printing propertiesConfig.getMessage() every 10 seconds. Therefore, I see a series of "1" in my log as myapp starts.

Right after, I change my reload.message ConfigMap's property to "2". What happens?

  • in less than one second the 'event' is triggered, and k8s calls my /actuator/refresh Spring Boot endpoint;
  • I still see "1" in the log, because of....
  • /config/application.yml (mounted volume) takes ~10s to update, then I can see reload.message=2 in there
  • 'refresh' happened just a few seconds ago, when the volume was not updated yet!

In addition, I tried other combos: mode polling, strategy restart_context, and so on. But... I definitely do want event+refresh! It's the required solution for our use case.

My question:

  • can I set some kind of "delay" for the refresh event, in order to give the volume the needed time to sync w the ConfigMap?
  • can I configure my ConfigMaps inside deployment without using volumeMounts at all? (if I remove config map volume now, Spring simply doesn't pick up the properties from ConfigMap)

Solution

  • The application.yml file in the project and the one described in the configmap conflict, somehow.

    I fixed that by renaming the one in the project to bootstrap.yml