Search code examples
amazon-web-servicesamazon-s3amazon-amiaws-marketplace

Access key in EC2 instance does not exist

I've created a new IAM Role called "AMIRole" and I've attached "AmazonS3FullAccess" policy to it. Now I launch an EC2 instance with this IAM role.

When I am SSH'd into the instance I can get access to the AWS Access key and Secret key like this:

curl http://169.254.169.254/latest/meta-data/iam/security-credentials/AMIRole
{
  "Code" : "Success",
  "LastUpdated" : "2016-12-21T16:05:26Z",
  "Type" : "AWS-HMAC",
  "AccessKeyId" : "ABCDEFJK",
  "SecretAccessKey" : "SECRET",
  "Token" : "TOKEN",
  "Expiration" : "2016-12-21T22:38:16Z"
}

However, when I try to create a bucket with this role, I get an error that this AWS KEY does not exist. 

>>> import boto3
>>> s3 = boto3.resource("s3", aws_access_key_id="ABCDEFJK", aws_secret_access_key="SECRET")
>>> s3.create_bucket(Bucket="something-specific-uuid")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
  File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 159, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 494, in _make_api_call
    raise ClientError(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the CreateBucket operation: The AWS Access Key Id you provided does not exist in our records.

My goal is to create an AMI that has rotating Access/secret keys so that I can put the AMI on AWS Marketplace.

Solution

  • A couple things about this:

    1. It's the role attached to an instance that grants it specific resource policies and access to different things. The AMI does not matter in this regard. If you put your AMI onto the AWS Marketplace, users will not have access to the same role you're currently using.

    2. The metadata service returns AccessKeyId, SecretAccessKey, and Token. All three of those must be specified if you're trying to use those credentials, so you'll have to specify something like this for boto3:

      import boto3

client = boto3.client(
    's3',
    aws_access_key_id=ACCESS_KEY,
    aws_secret_access_key=SECRET_KEY,
    aws_session_token=SESSION_TOKEN,
)

    3. When running code on the instance using an SDK like boto3, you don't need to provide the credentials at all. boto3 will use the IAM role's credentials from the metadata service should it fail to find any other form of credentials

      Note that if you've launched an EC2 instance with an IAM role configured, there's no explicit configuration you need to set in boto3 to use these credentials. Boto3 will automatically use IAM role credentials if it does not find credentials in any of the other places listed above.

      So for you, if you're running code directly on the instance using boto3, you could just create the boto3.resource without needing to specify access keys or secrets.