I followed this tutorial to link my aws s3 bucket with my Heroku app.
In my s3 bucket I have a zip file called model.zip
. I need to use this model in my app (Python). So after linking the bucket successfully according to the tutorial, I did the following:
s3 = boto3.client('s3')
s3.download_file('my-models-bucket', 'model.zip', 'model.zip')
But when I run it on Heroku, I get a 403 error at the s3.download_file
line:
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Do I need to specify the credentials in boto3.client()
? I don't want to upload any credentials, so how should it be done? Or should I somehow use the environment variables I set up while linking s3 according to the tutorial? I.e. the tutorial says to do the following:
heroku config:set AWS_ACCESS_KEY_ID=MY-ACCESS-ID AWS_SECRET_ACCESS_KEY=MY-ACCESS-KEY
Adding config vars and restarting app... done, v21
AWS_ACCESS_KEY_ID => MY-ACCESS-ID
AWS_SECRET_ACCESS_KEY => MY-ACCESS-KEY
Does that mean that I now have access to AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
in my Python code and I should read them using os.environ.get("AWS_ACCESS_KEY_ID")
or something?
Or maybe there's something I should do on Aws?
Answering myself after finding the solution...
I simply had to add a Policy to my s3 bucket, by going to the bucket > Permissions > Edit bucket policy > Policy generator > Fill up the fields and desired permissions (I only picked GetObject), generate the Policy, copy and paste it under Bucket Policy. Here's mine:
{
"Version": "2012-10-17",
"Id": "Policy1234567",
"Statement": [
{
"Sid": "Stmt1234567",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:user/my-user"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my_bucket_name/*"
}
]
}
Now everything is working.