Search code examples
springspring-bootspring-cloudspring-cloud-confighashicorp-vault

Spring Boot & Vault: incomplete context initialization issue


I've faced an issue that on rare occasions (it might take dozens of restarts) Spring doesn't initialize all properties correctly.

I define the bean of CbKafkaConsumerConfig (my custom bean) type and check its state in the thread that was created by a method that is marked as @EventListener(ApplicationReadyEvent.class), so I expect it to be completely initialized by this point. However, this is what I see:

Bean state in debug

Values that I expected to be filled are left with placeholders. Here's how they are defined in application.properties file. (And I've checked the spelling - it's correct, otherwise it would fail every time, not occasionally)

config-bean-prefix.msg-topics=${cb.kafka.tc-topic}
config-bean-prefix.unexpected-error-topic=${cb.kafka.unexpected-errors-topic}

These properties are defined in Vault and I expected them to be fetched and set with the power of Spring Cloud Vault. Here you can see that Vault is present as a property source AND that these properties are populated there.

Vault properties are in the context Missing properties are present in the Vault

At the same time, in the context there are other beans of the same type CbKafkaConsumerConfig that are referring to these properties and yet it resolved fine for them.

Here's how the bean is defined

@Bean({"myBean"})
@ConfigurationProperties(
    prefix = "config-bean-prefix"
)
public CbKafkaConsumerConfig myBeanConsumer() {
    return new CbKafkaConsumerConfig();
}

And the bean itself:

@Data
public class CbKafkaConsumerConfig extends CbKafkaBaseConfig {

  @NotNull
  @Size(
      min = 1
  )
  private Collection<String> msgTopics;
  @NotNull
  private String unExpectedErrorTopic;
}

We're using Spring Boot 2.2.x however this issue is also present for Spring Boot 2.1.x. It's not specific for this type of beans, other might fail as well while being correctly set in Vault. What could be the reason of such unpredictable behavior and what I should look into?


Solution

  • Turns out by default spring cloud vault is not simply fetching properties on start, every so often it's updating them. While updating, there's a short time window when properties were already deleted from property source in the context, but not filled with the new ones and it might actually happen during context initialization (super questionable behavior in my opinion) causing some beans being corrupted.

    If you don't want properties to be updated in runtime just set spring.cloud.vault.config.lifecycle.enabled to false