Search code examples
amazon-web-servicesamazon-s3amazon-sns

Why does S3 file upload not trigger event to SNS topic?

I want a certain HTTPS service to be called every time a file has been uploaded to an S3 bucket.

I have created the S3 bucket and a SNS topic with a verified subscription with the HTTPS service as an endpoint. I can publish a message on the SNS topic via the AWS UI, and see that the HTTPS service is called as expected. On the S3 bucket I created an Event, which should link the bucket and the topic. On my first attempt I got an error because the bucket was not allowed to write to the topic, so c.f. the documentation, I changed the topic access policy to:

{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "sns:Publish",
      "Resource": "arn:aws:sns:eu-central-1:TOPIC_ID:OrderUpdates",
      "Condition": {
        "StringEquals": {
          "AWS:SourceAccount": "ACCOUNT_ID"
        },
        "ArnLike": {
          "AWS:SourceArn": "arn:aws:s3:*:*:*"
        }
      }
    }
  ]
}

where TOPIC_ID is the topic owner id which can be seen when the topic is shown in the AWS UI, and the ACCOUNT_ID is the account id shown under account settings in the AWS UI. This change in the topic access policy allowed me to create the event on the bucket: enter image description here

When I call the API method getBucketNotificationConfiguration I get:

{
    "TopicConfigurations": [
        {
            "Id": "OrderFulfilled",
            "TopicArn": "arn:aws:sns:eu-central-1:TOPIC_ID:OrderUpdates",
            "Events": [
                "s3:ObjectCreated:*"
            ]
        }
    ],
    "QueueConfigurations": [],
    "LambdaFunctionConfigurations": []
}

But the HTTPS service is not called. What am I missing in this setup, that will trigger the HTTPS service to be called by the SNS topic subscription every time a file is uploaded to the S3 bucket?

Thanks, -Louise

Solution

  • To reproduce this situation, I did the following:

    • Created an Amazon SNS topic and subscribed my phone via SMS (a good way to debug subscriptions!)
    • Created an Amazon S3 bucket with an Event pointing to the Amazon SNS topic

    I received this error message:

    Unable to validate the following destination configurations. Permissions on the destination topic do not allow S3 to publish notifications from this bucket.

    • I then added the policy you show above (adjusted for my account and SNS ARN)
    • This allowed the Event to successfully save

    Testing

    • I then tested the event by uploading a file to the S3 bucket.
    • I received an SMS very quickly

    So, it would appear that your configuration should successfully enable a message to be sent via Amazon SNS. This suggests that the problem lies with the HTTPS subscription, either from sending it from SNS or receiving it in the application.

    I recommend that you add an Email or SMS subscription to verify whether Amazon SNS is receiving the topic and forwarding it to subscribers. If this works successfully, then you will need to debug the receipt of the message in the HTTPS application.