AWS SAM CLI DEPLOYMENT - Issues Parsing Out of the Box secret = get_secret_value_response['SecretString']

Question

I am very green when it comes to Python - so bare with me. Using the out of the Box Python3 Sample Code provided by AWS to return the 'SecretString' from the AWS: Secrets Manager Service.

No issues there .. I get the returned object (note I have blanked out some details)

{"username":"postgres","password":"XXXXXXXXX","engine":"postgres","host":"srdataset.XXXXXXXXX.ap-southeast-2.rds.amazonaws.com","port":5432,"dbInstanceIdentifier":"srdataset"}  

detail are all correct.

I am then using json.loads() to parse the above into my next function so I can extract the details like so

    # request details
    login_details = get_secret("pg_srdataset_login_details")

    # load json
    y = json.loads(login_details)

    # extract result is a Python dictionary:
    print(y["username"])

This again all works fine in my IDE (PyCharm). I can run the code, build in a Docker container .. and I then use the PyCharm AWS SAM CLI to deploy the code to the cloud .. no issues.

However when I test the function in AWS the code bugs out on the line y = json.loads(login_details) step.

the error being ..

{
  "errorMessage": "Expecting value: line 1 column 1 (char 0)",
  "errorType": "JSONDecodeError",
  "stackTrace": [
    "  File \"/var/task/update_sp_changes.py\", line 229, in lambda_handler\n    y = json.loads(login_details)\n",
    "  File \"/var/lang/lib/python3.8/json/__init__.py\", line 357, in loads\n    return _default_decoder.decode(s)\n",
    "  File \"/var/lang/lib/python3.8/json/decoder.py\", line 337, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n",
    "  File \"/var/lang/lib/python3.8/json/decoder.py\", line 355, in raw_decode\n    raise JSONDecodeError(\"Expecting value\", s, err.value) from None\n"
  ]
}

To test this I also copied the JSON 'SecretString' returned from AWS, hard coded it as a variable, then passed this variable directly into the y = json.loads(login_details) step. Tested again and works a treat.

What am I doing wrong - how can I work around this issue.

Answer 1

For reference, I'd share Lambda print debug code.

Preparation:

• from the Lambda-py38 initial state, SecretsManagerReadWrite policy is attached to Lamda
• set SecretId and VersionId

Output:

• json.loads() works for plaintext
• print() output almost the same for plain_text:str and secret_dict:dict. (Difference is quotation:single/double. Which are treated as the same string when copy-pasted.)

import json
import boto3

def lambda_handler(event, context):

    client = boto3.client('secretsmanager')

    response = client.get_secret_value(
        SecretId='arn:aws:secretsmanager:xxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
        VersionId='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
        VersionStage='AWSCURRENT'
    )

    print("type of response: ", type(response))
    print("response: ", response)

    plaintext = response['SecretString']
    print("type of plaintext: ", type(plaintext))
    print("plaintext: ", plaintext)

    secret_dict = json.loads(plaintext)
    print("type of secret_dict: ", type(secret_dict))
    print("secret_dict: ", secret_dict)

    return 200