Search code examples
pythonamazon-s3heroku

How to set up aws s3 with Heroku?


I followed this tutorial to link my aws s3 bucket with my Heroku app.

In my s3 bucket I have a zip file called model.zip. I need to use this model in my app (Python). So after linking the bucket successfully according to the tutorial, I did the following:

s3 = boto3.client('s3')
s3.download_file('my-models-bucket', 'model.zip', 'model.zip')

But when I run it on Heroku, I get a 403 error at the s3.download_file line:

botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

Do I need to specify the credentials in boto3.client()? I don't want to upload any credentials, so how should it be done? Or should I somehow use the environment variables I set up while linking s3 according to the tutorial? I.e. the tutorial says to do the following:

heroku config:set AWS_ACCESS_KEY_ID=MY-ACCESS-ID AWS_SECRET_ACCESS_KEY=MY-ACCESS-KEY
Adding config vars and restarting app... done, v21
  AWS_ACCESS_KEY_ID     => MY-ACCESS-ID
  AWS_SECRET_ACCESS_KEY => MY-ACCESS-KEY

Does that mean that I now have access to AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in my Python code and I should read them using os.environ.get("AWS_ACCESS_KEY_ID") or something?

Or maybe there's something I should do on Aws?


Solution

  • Answering myself after finding the solution...

    I simply had to add a Policy to my s3 bucket, by going to the bucket > Permissions > Edit bucket policy > Policy generator > Fill up the fields and desired permissions (I only picked GetObject), generate the Policy, copy and paste it under Bucket Policy. Here's mine:

    {
        "Version": "2012-10-17",
        "Id": "Policy1234567",
        "Statement": [
            {
                "Sid": "Stmt1234567",
                "Effect": "Allow",
                "Principal": {
                    "AWS": "arn:aws:iam::123456789012:user/my-user"
                },
                "Action": "s3:GetObject",
                "Resource": "arn:aws:s3:::my_bucket_name/*"
            }
        ]
    }
    

    Now everything is working.