Get temp creds in Lambda that live longer than the lambda

Question

So, I'm inside an AWS lambda function. I need to return temp creds from an API endpoint, so that an upload can be performed from browser directly to S3.

The files can be very large, hundreds of gigabytes, so the creds need to last a long time. What's the easiest way to get these type of creds inside a lambda?

Answer 1

The short answer is that you need to assume a role, as I describe in this blog post. A key part of that post is using a session policy to scope the assumed role to a single key on S3.

However, if it takes more than an hour to upload the file, that solution won't work as written, because a Lambda can't assume another role with a requested duration > one hour (see role chaining), a limit that can't be increased.

This means that you need to create a user that can assume the role, and make that user's long-term credentials available to the Lambda (typically via Secrets Manager). Once you've retrieved those credentials, use them to create an STS client (you don't say what language you're using, and I typically use Python, so that's what's shown):

sts_client = boto3.client(
    'sts',
    aws_access_key_id=stored_access_key,
    aws_secret_access_key=stored_secret_key)

Then with those credentials you can assume a role that can write to the file. In following the blog post, the base role has permissions to write to any file on S3, and the session policy limits the assumed role to the specific file:

session_policy = json.dumps({
    'Version': '2012-10-17',
    'Statement': [
        {
            'Effect': 'Allow',
            'Action': 's3:PutObject',
            'Resource': f"arn:aws:s3:::{BUCKET}/{KEY}"
        }
    ]
})

response = sts_client.assume_role(
    RoleArn=ASSUMABLE_ROLE_ARN,
    RoleSessionName="example",
    Policy=session_policy,
    DurationSeconds=12 * 3600
)

# these are the credentials that you'd pass to the client application

limited_access_key = response['Credentials']['AccessKeyId']
limited_secret_key = response['Credentials']['SecretAccessKey']
limited_session_token = response['Credentials']['SessionToken']

12 hours is enough time to transfer 500 GB over a 100 mbps connection. If you need to more time than that, then you'll have to create an actual user and return its credentials. You can attach an inline policy to this user to limit its access to the single file (serving the same purpose as the session policy in this example). But since you're limited to 5,000 IAM users in an account, this is not something that you want to do on a regular basis.