Search code examples
springspring-vaultvault

Vault login token expiring unexpectedly

We operate a containerized Spring Boot application. In order to access third party APIs, secrets to those APIs are kept in a Vault instance. Our application connects to Vault via Spring Vault Core using token authentication:

spring:
  cloud:
    vault:
      fail-fast: true
      ...
      authentication: token
      session:
        lifecycle:
          refresh-before-expiry: 15s
          expiry-threshold: 25s

The token is handed over to our application on startup via an environment variable spring.cloud.vault.token.

The token itself is created as a periodic service token using vault token create -policy=<some policy> -period 4h. It shows renewable=true and has no explicit max TTL. As such, it should never expire if properly renewed during application lifetime. This is handled automatically by Spring Vault's LifecycleAwareSessionManager.

Now in testing as well as in production environments, the generated token in fact expires from time to time even though being renewed before expiry. The remaining TTL that is returned to Spring Vault's renewal attempt shows the TTL is not reset but continues to decrease until the token expires.

Does anyone have a clue why this might happen? Or else: what might cause a periodic service token to expire even though it is properly renewed?

Solution

  • Two different notes:

    • First, child token cannot outlive it's parent. If the parent token expires, the child token regardless of it's TTL left, will also expire. If you need the token to outlive its parent you need to provide -orphan param when creating the token.
    • Second, renewable does not mean, you can renew forever, you can keep extending the life of a token UP TO the "max ttl". But that's it. The token will expire. "Renew" does not get you a now token, it simply extends the life of the current TTL. You still cannot get past the max-ttl age limit. My suggestion is not to use service tokens, switch to batch tokens (still will run into maxttl issues) or even better an auth system (approle for example) to get your apps access.

    If you're in secure environment (kub pods, or locked VM for example) consider using the Vault Agent. It'll help you to keep your session alive.