Search code examples
amazon-web-servicesaws-lambdaamazon-kms

InvalidChiperException for one lambda, but not the other


db_password = boto3.client('kms').decrypt(CiphertextBlob=b64decode(os.environ['DB_PASSWORD']))['Plaintext']

services_region = 'us-east-1'    
db_password = boto3.client('kms', region_name=services_region).decrypt(CiphertextBlob=b64decode(os.environ['DB_PASSWORD']))['Plaintext']

I have tried both of the above code, I have verified that both lambda functions are using the same VPC, private subnets, and that the security groups for both are added to the kms endpoint. They both use the same exact role. They both use the same S3 zip package and they both use the same line in the code for the .decrypt (I copied and pasted the code, it is the second one above).

One of them works without issue, the other is throwing the following:

An error occurred (InvalidCiphertextException) when calling the Decrypt operation: : InvalidCiphertextException
Traceback (most recent call last):
  File "/var/task/awsfinbi_workdocs_api_pull/bin/api_pull.py", line 143, in lambda_handler
    get_data()
  File "/var/task/awsfinbi_workdocs_api_pull/bin/api_pull.py", line 100, in get_data
    db_load(df, s)
  File "/var/task/awsfinbi_workdocs_api_pull/bin/api_pull.py", line 122, in db_load
    raise e
  File "/var/task/awsfinbi_workdocs_api_pull/bin/api_pull.py", line 112, in db_load
    db_password = boto3.client('kms',region_name=services_region).decrypt(CiphertextBlob=b64decode(os.environ['DB_PASSWORD']))['Plaintext']
  File "/var/runtime/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/var/runtime/botocore/client.py", line 635, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.InvalidCiphertextException: An error occurred (InvalidCiphertextException) when calling the Decrypt operation:

The only difference between the two is the passwords are different. However, I tested it with my password which is different to the programmatic user and it seems to make no difference.

the permissions tab on the lambda shows kms:* Allow: All resources

I have tried creating a key and using it, I have tried using the all the other keys as well. All throw this error.

Anything I might have missed to figure out this issue?


Solution

  • Well somehow I stumbled on an answer.

    1 things about my code, that wasn't posted, is I have a for loop that was calling the decrypt more than once. Apparently that should never be done. Thus I moved the code for the decrypt out of the for loop.

    Even removing it from the for loop didn't fix it, but it worked when I made it two lines as follows:

    ENCRYPTED = os.environ['DB_PASSWORD']
    db_password = boto3.client('kms').decrypt(
        CiphertextBlob=b64decode(ENCRYPTED),
        EncryptionContext={'LambdaFunctionName': os.environ['AWS_LAMBDA_FUNCTION_NAME']}
    )['Plaintext'].decode('utf-8')
    

    I had previously tried the context and it didn't help at the time with it all one line. I am not sure why it didn't like having it in all one line, but the above is working.