Assigning policy to S3Bucket in order to be used only by a CloudFront Distribution

Question

This script shows how I create a CloudFront origin access identity, a bucket that will hold my webapp and how I assign the bucket policy in order to only allow access to the bucket from the CloudFront distribution.

Having this scenario, what it is really surprising (and annoying) is the fact that this code works If I debug it line by line within Eclipse but If I try to launch it without going line by line (i.e. setting a breakpoint just after the policy assignment), then the below exception appears...

Hope someone can help!

String myBucket = transferManager.getAmazonS3Client().createBucket(new CreateBucketRequest("my-bucket-name")).getName();

CloudFrontOriginAccessIdentity myOAI = cloudFrontClient.createCloudFrontOriginAccessIdentity(
                        new CreateCloudFrontOriginAccessIdentityRequest().withCloudFrontOriginAccessIdentityConfig(
                                new CloudFrontOriginAccessIdentityConfig().withCallerReference(UUID.randomUUID().toString()).withComment("myOAI"))).getCloudFrontOriginAccessIdentity();

//*ATTEMPT 1: Using canonical user Id*
transferManager.getAmazonS3Client().setBucketPolicy(myBucketName, new Policy().
withId("MyPolicyForCloudFrontPrivateContent").
withStatements(new Statement(Effect.Allow).
withId("Grant CloudFront Origin Identity access to support private content").
withActions(S3Actions.GetObject).
withPrincipals(new Principal("CanonicalUser:" + myOAI.getS3CanonicalUserId())).
withResources(new S3ObjectResource(myBucketName,"*"))).toJson());

//*ATTEMPT 2: Using OAI id*
transferManager.getAmazonS3Client().setBucketPolicy(myBucketName, new Policy().
withId("MyPolicyForCloudFrontPrivateContent").
withStatements(new Statement(Effect.Allow).
withActions(S3Actions.GetObject).
withPrincipals(new Principal("arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity " + myOAI.getId())).
withResources(new S3ObjectResource(myBucketName,"*"))).toJson());

//*ATTEMP 3: HARDCODING THE POLICY*
String myPolicy = "{\"Version\":\"2012-10-17\",\"Id\":\"PolicyForCloudFrontPrivateContent\",\"Statement\":[{\"Sid\":\" Grant a CloudFront Origin Identity access to support private content\",\"Effect\":\"Allow\",\"Principal\":{\"CanonicalUser\":\"" + myOAI.getS3CanonicalUserId() + "\"},\"Action\":\"s3:GetObject\",\"Resource\":\"arn:aws:s3:::" + myBucketName + "/*\"}]}";
transferManager.getAmazonS3Client().setBucketPolicy(myBucketName, myPolicy);


//*ERROR MESSAGE*

Exception in thread "main" com.amazonaws.services.s3.model.AmazonS3Exception: Invalid principal in policy (Service: Amazon S3; Status Code: 400; Error Code: MalformedPolicy; Request ID: XXXXXXXXXXXXX), S3 Extended Request ID: YYYYYYYYYYYYYYYYYYYYYY+XXXXXXXXXXXXXXXXXXXXXXX=
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1088)
    at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:735)
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:461)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:296)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3737)
    at com.amazonaws.services.s3.AmazonS3Client.setBucketPolicy(AmazonS3Client.java:2372)
    at com.myapp.services.DeploymentService.applyVersion(DeploymentService.java:234)
    at com.myapp.services.DeploymentService.launch(DeploymentService.java:3553)
    at com.myapp.EntryPoint.main(EntryPoint.java:35)

Answer 1

Found the problem...

It looks like when you create a Cloudfront Origin Access Identity (OAI) and try to assign it into a bucket policy inmediately the error appears because that new OAI change is not propagated inmediately.

A valid workaround is to implement a retry condition policy:

class CloudFrontRetryCondition implements RetryCondition {
    @Override
    public boolean shouldRetry(AmazonWebServiceRequest originalRequest, AmazonClientException exception, int retriesAttempted) {
        if(exception instanceof AmazonS3Exception) {
            final AmazonS3Exception s3Exception = (AmazonS3Exception) exception;
            return  s3Exception.getStatusCode() == 400 &&
                    s3Exception.getErrorCode().equals("MalformedPolicy") &&
                    s3Exception.getErrorMessage().equals("Invalid principal in policy") &&
                s3Exception.getAdditionalDetails().get("Detail").contains("arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity");
        } else {
            return false;
        }
    }
}