I would like to be able to access (read) another account's S3 bucket in my AWS ParallelCluster post-install script.
The bucket is shared with my account through the Bucket's ACL settings using my canonical Account ID. I can access the bucket from my local machine through aws s3 cp.
What I'd like to do is specify the other account's bucket via ParallelCluster's s3_read_resource setting. However, it seems like I can only specify my own account's buckets.
Is there any workaround? The other account's bucket is set so that the requester pays the network traffic costs. Anonymous access to the bucket is therefore not possible. The purpose is to provide data to authenticated third-parties to use with their clusters.
After playing around with complex workarounds, it turns out that this is quite simple via additional_iam_policies:
additional_iam_policies = arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
This way, the created instances will be able to read all S3 buckets that your account can access, also those shared with you by other accounts.