Malformed S3 policy from Terraform

Question

Looks like I'm getting a malformed policy as part of my Terraform deployment and am not seeing any issue(s) with it while looking it over, all the Principals seem to be in place and nothing looks out of the ordinary. Can someone pick out why this policy is being considered malformed?

Here's the TF output that is throwing

module.s3-bucket.aws_s3_bucket_policy.bucket_policy: Creating...
  bucket: "" => "the_s3_bucket"
  policy: "" => "{\n  \"Version\":\"2012-10-17\",\n    \"Statement\":[{\n      \"Principal\": \"*\",\n      \"Action\":[\n        \"s3:GetBucketLocation\",\n        \"s3:ListAllMyBuckets\"\n      ],\n        \"Resource\":[\n          \"arn:aws:s3:::*\"\n      ],\n    \"Effect\":\"Allow\"\n  }, {\n    \"Principal\": \"*\",\n    \"Action\":[\n      \"s3:ListBucket\",\n      \"s3:ListBucketMultipartUploads\"\n    ],\n    \"Resource\":[\n      \"arn:aws:s3:::the_s3_bucket\"\n    ],\n    \"Effect\":\"Allow\"\n    }, {\n      \"Principal\": \"*\",\n    \"Action\":[\n      \"s3:GetObject\",\n      \"s3:AbortMultipartUpload\",\n      \"s3:ListMultipartUploadParts\",\n      \"s3:DeleteObject\",\n      \"s3:PutObject\",\n      \"s3:GetObjectAcl\",\n      \"s3:PutObjectAcl\"\n    ],\n    \"Resource\":[\n      \"arn:aws:s3:::the_s3_bucket/*\"\n    ],\n    \"Effect\":\"Allow\"\n  }]\n}\n"

The (non-verbose) Error:

Error applying plan:

1 error(s) occurred:

* module.s3-bucket.aws_s3_bucket_policy.bucket_policy: 1 error(s) occurred:

* aws_s3_bucket_policy.bucket_policy: Error putting S3 policy: MalformedPolicy: Policy has invalid action
    status code: 400, request id: DCAEB510D9FE431C, host id: FwZ187PM8RKZljAZGo1578UvsgW3kwtZpaI2Mom46lu7Jr+NV9Jum8txjsfwdJw0jm9ct0awRWk=

Here's a pretty print of what it contains

{
  "Version":"2012-10-17",
    "Statement":[{
      "Principal": "*",
      "Action":[
        "s3:GetBucketLocation",
        "s3:ListAllMyBuckets"
      ],
        "Resource":[
          "arn:aws:s3:::*"
      ],
    "Effect":"Allow"
  }, {
    "Principal": "*",
    "Action":[
      "s3:ListBucket",
      "s3:ListBucketMultipartUploads"
    ],
    "Resource":[
      "arn:aws:s3:::the_s3_bucket"
    ],
    "Effect":"Allow"
    }, {
      "Principal": "*",
    "Action":[
      "s3:GetObject",
      "s3:AbortMultipartUpload",
      "s3:ListMultipartUploadParts",
      "s3:DeleteObject",
      "s3:PutObject",
      "s3:GetObjectAcl",
      "s3:PutObjectAcl"
    ],
    "Resource":[
      "arn:aws:s3:::the_s3_bucket/*"
    ],
    "Effect":"Allow"
  }]
}

Update

So evidently this has something to do with the Principal field, after running this through AWS Console I get the error

This policy contains the following error: Has prohibited field Principal For more information about the IAM policy grammar, see AWS IAM Policies 

Answer 1

The s3:ListAllMyBuckets permission cannot be applied to an S3 bucket policy and instead is an IAM policy permission. You must also directly specify the bucket that the policy is attached to in the bucket policy rather than use "Resource": ["*"] or some other wildcard for the bucket.

IAM policies and S3 bucket policies are closely related and overlap heavily but you should consider them as a further level of control that can sometimes be necessary. Unfortunately the S3 permissions model is very confusing and has multiple layers of adding access control, from bucket ACLs and object ACLs (including fully configurable ones and canned ones) to bucket policies and also straight IAM permissions that are attached to an IAM user or role.

Most of the time you are going to be best off using the canned ACLs and then use IAM user policies to control who can perform extra actions not allowed by the ACL. There's further explanation of these in the User Guide.