How do I fix a "(403) when calling the HeadObject operation: Forbidden"

Question

I am attempting to run a spark AWS Glue Job.

I have uploaded a .py file to a S3 bucket, lets call it jobfiles_s3_bucket.

When the glue job runs, it tries to copy the .py from the jobfiles bucket to a temp directory:

S3 Copy Started: 
SRC: s3://<insert_bucket_here>/glue-job.py 
DEST: /tmp/g-8aa076b857cf7854bfe423a40d5c07863eb091a9-2843067097771260387/script_2024-01-21-00-17-12.py
Specifying ap-southeast-2 while copying script.

S3 copy with region specified failed. Falling back to not specifying region.

I get this output:

fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

Error downloading script: fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

The glue job has an IAM role with the following:

   {
      "Sid" : "AccessPointRWDAccessands3access",
      "Effect" : "Allow",
      "Action" : [
        "s3:*Object",
        "s3:Put*",
        "s3:List*",
        "s3:Get*",
        "s3:Delete*"
      ],

With Resources to both the temp directory and the jobfiles bucket, but I still get the error?

What else could cause this issue? I'm not sure what else could be wrong?

The glue job should be able to run the .py file, as it has access to the all the s3 areas it requires to collect the .py file and has all the permissions required to run.

Answer 1

The posted IAM Role looks a bit strange. It is missing the "Ressource" and "Principal" section e.g.

"Principal": {
"AWS": ["arn:aws:iam::<AccountNumber>:root",]
},
...
"Resource": ["arn:aws:s3:::<bucketName>/*"]

If you have allowed access to the resource, make sure that you include access to the encryption key if you have encrypted the bucket. These two seem to be the most likely causes in my eyes.

For more troubleshooting you can see the documentation Troubleshoot Access Denied (403 Forbidden) errors in Amazon S3 which also includes what I have mentioned above.