How to store & reuse response data from a CloudFormation Custom Resource Lambda function within the stack?

Question

I'm designing an AWS Lambda function for a Cloudformation Custom Resource as per this documentation.

Upon creation of the Custom Resource, the associated Lambda function should (a) create an external resource, and (b) return the ID associated with that external resource.

Upon the update or deletion of the Custom Resource, the associated Lambda function should take the ID of the existing external resource as a parameter/property, and update/delete that external resource accordingly.

To do so, the CloudFormation stack needs to store that ID returned by the Lambda upon the creation of the Custom Resource. Ideally, it should also associate that ID with the Custom Resource itself.

How can this be achieved?

Example code for the scenario in question:

Resources:
  CustomResource:
    Type: AWS::CloudFormation::CustomResource
    Properties:
      ServiceToken:
        Fn::GetAtt: Lambda.Arn
      ID: <see-below>
  Lambda:
    Type: AWS::Lambda::Function
    Properties:
      ...

(where ID doesn't exist before the creation of CustomResource, but it needs to be populated by the result of Lambda upon creation)

Answer 1

You are confusing several things.

Upon the update or deletion of the Custom Resource, the associated Lambda function should take the ID of the existing external resource as a parameter/property, and update/delete that external resource accordingly.

This is handled via the PhysicalResourceId property. When you create a resource, you are required to return this in the response. CloudFormation will then pass this value to the Lambda for an update or delete.

The contents of that ID are opaque to CloudFormation, and you can store up to 1KB in it. So you can get quite creative with what you store in it. For example, you could store a URL that you use to invoke some third-party web-service.

ID: <see-below>

In this case, you're specifying a property that you pass to the custom resource Lambda. You will find it in the ResourceProperties property of the event. The example that you linked to doesn't show any custom properties, but if you log the invocation you'll see ID there.

return the ID associated with that external resource

You probably shouldn't return the ID of the external resource; it should remain an opaque identifier. But if you do want to return it, then you'll need to add a Data element to the custom resource response. This is a JSON object that contains resource "attributes" that can be retrieved using Fn::GetAtt.

For more information on custom resources, I recommend this doc, which specifies all of the fields in the request and response object.

Also, if you're using the cfnresponse module (which appears any many of the AWS official docs), you shouldn't: it relies on a "vendored" copy of requests in the boto3 library, and that's been deprecated for at least two years. At some point you'll find that your templates stop working (and failed custom resources can be very difficult to recover from).