Disable Spring Boot Data MongoDB retryable writes in AWS ECS Fargate with CloudFormation

Question

I'm using CloudFormation to deploy a Spring Boot application in ECS Fargate from a Docker image. It uses Spring Boot Data MongoDB to connect to AWS DocumentDB. Since everything is self-contained in the CloudFormation template (i.e. everything is deployed via the template—no copying and pasting or manual provisioning), I inject the DB cluster in the task container definition, along with the username+password from Secrets Manager:

          Environment:
            - Name: SPRING_DATA_MONGODB_HOST
              Value: !GetAtt DbCluster.Endpoint
            - Name: SPRING_DATA_MONGODB_PORT
              Value: !GetAtt DbCluster.Port
          Secrets:
            - Name: SPRING_DATA_MONGODB_USERNAME
              ValueFrom: !Sub "${DbCredentials}:username::"
            - Name: SPRING_DATA_MONGODB_PASSWORD
              ValueFrom: !Sub "${DbCredentials}:password::"

I just discovered that DocumentDB doesn't support retryable writes. So when my ECS task spins up, it says:

com.mongodb.MongoCommandException: Command failed with error 301: 'Retryable writes are not supported' on server my-db-cluster.cluster-xxxxxxxxxxxx.us-east-1.docdb.amazonaws.com:27017. The full response is {"ok": 0.0, "code": 301, "errmsg": "Retryable writes are not supported", "operationTime": {"$timestamp": {"t": xxxxxxxxxx, "i": 1}}}

The AWS documentation and various other answers are saying to set retryWrites=False by switching to the full SPRING_DATA_MONGODB_URI with the connection URI format like this:

mongodb://<username>:<password>@my-db-cluster.cluster-xxxxxxxxxxxx.us-east-1.docdb.amazonaws.com:27017/database?retryWrites=False

That would be something like this in my task container definition in CloudFormation:

            - Name: SPRING_DATA_MONGODB_URI
              Value: !Sub "mongodb://${DbCluster.Endpoint}:${DbCluster.Port}/?retryWrites=False"

But it appears that defining the connection URI overrides the username and password environment variables. So now how do I inject the username+password from Secrets Manager into the connection URI in CloudFormation?

Or is there some other way to disable retryable writes in Spring Boot Data MongoDB that still allows me to inject credentials from Secrets Manager?

Answer 1

Whenever you specify the individual Spring Boot Data MongoDB environment variables such as SPRING_DATA_MONGODB_PASSWORD (which will be accessible internally via configuration properties such as spring.data.mongodb.password), Spring will construct the connection URL dynamically. But if you specify the connection URI environment variable SPRING_DATA_MONGODB_URI, Spring will ignore the individual components, including the username and password ECS has injected from Secrets Manager, as noted in the question.

But there is a workaround to force Spring to pull in the secret credentials you've injected even via the connection URI. Spring Boot provides a facility for interpolating configuration properties dynamically using property placeholders, similar to CloudFormation references except that these references are replaced at runtime. So it's possible to go ahead and inject the username and password, and then use placeholder in the actual connection URI, like this:

          Environment:
            - Name: SPRING_DATA_MONGODB_HOST
              Value: !GetAtt DbCluster.Endpoint
            - Name: SPRING_DATA_MONGODB_PORT
              Value: !GetAtt DbCluster.Port
            - Name: SPRING_DATA_MONGODB_URI
              Value: mongodb://${spring.data.mongodb.username}:${spring.data.mongodb.passsword}@${spring.data.mongodb.host}:${spring.data.mongodb.port}/?retryWrites=False
          Secrets:
            - Name: SPRING_DATA_MONGODB_USERNAME
              ValueFrom: !Sub "${DbCredentials}:username::"
            - Name: SPRING_DATA_MONGODB_PASSWORD
              ValueFrom: !Sub "${DbCredentials}:password::"

Based on the recommendations the AWS DocumentDB console provides, you might even want to use this extended connection URI:

mongodb://${spring.data.mongodb.username}:${spring.data.mongodb.password}@${spring.data.mongodb.host}:${spring.data.mongodb.port}/?replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false

Now username and password are injected into environment variables at deploy time, and Spring Boot itself will plug those values into the connection URI for you at runtime.

There is a tiny caveat that is necessary to mention but otherwise inconsequential. If the password were to contain the plain @ or : characters, it would make the username:password@… portion of the URI ambiguous, so MongoDB requires those to be URI encoded. Furthermore if you were to include %, that character would also need to be encoded because it's the first character in a URI encoding sequence. As Spring Boot has no way of dynamically URI-encoding configuration property references, it's easy enough to tell AWS Secrets Manager to exclude those three characters when generating a password. AWS::DocDB::DBCluster already prohibits the /, ", and @ characters, so really you're only removing two extra characters from those available.

Thus you might generate a AWS::SecretsManager::Secret like this in CloudFormation:

  DbCredentials:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: db-creds
      GenerateSecretString:
        SecretStringTemplate: '{"username": "somebody"}'
        GenerateStringKey: "password"
        PasswordLength: 99
        ExcludeCharacters: '/"@%:'

(Tip: Although DocumentDB claims to support passwords of 100 characters, in reality the underlying RDS will still complain if the password is over 99 characters in length.)

The Spring Boot Data MongoDB documentation says:

Username and password credentials used in XML-based configuration must be URL-encoded when these contain reserved characters, such as :, %, @, or ,. The following example shows encoded credentials: m0ng0@dmin:mo_res:bw6},Qsdxx@admin@database → m0ng0%40dmin:mo_res%3Abw6%7D%2CQsdxx%40admin@database See section 2.2 of RFC 3986 for further details.

However RFC 3986 § 3.2.1. User Information isn't so limited, and can actually contain: sub-delims from § 2.2.

userinfo = *( unreserved / pct-encoded / sub-delims / ":" )

Thus if we exclude the remaining delimiters, i.e. gen-delims, from § 2.2, that gives us the following, which is probably most correct and safest:

        # exclude `/"@` as per DocumentDB restrictions
        # exclude `%`, the URI encoding delimiter, so the password can be interpolated into a connection URI
        # exclude `:/?#[]@` `gen-delims` from [RFC 3986 § 2.2](https://www.rfc-editor.org/rfc/rfc3986.html#section-2.2)
        ExcludeCharacters: "/\"@%:?#[]"