Is it possible to trigger an AWS Fargate task upon an item being added to an SQS queue?

Question

For clarification, what I'm trying to do is fire off a Fargate task when theres an item in a specific queue. I've used this tutorial to get pretty much where I am. This worked fine but the problem I ran into was every file upload (the structure of the s3 bucket is s3_bucket_name/{unknown_name}/known_file_names) was resulting in a task being triggered and I only want/need it to trigger once per {unknown_name} . I've since changed my configuration to add an item to a queue when it detects a test_file.txt file. Is it possible to trigger a fargate task on a queue like this? If so, how?

Answer 1

SQS doesn't trigger or "push" messages to anything. As mentioned in the comments, AWS Lambda has an SQS integration that can automatically poll SQS for you and trigger a Lambda function with new messages, which you could use to create your Fargate tasks.

However I would recommend refactoring your Fargate task like this:

• Reconfigure the code running in your container to poll the SQS queue for messages.
• Run your task as an ECS service.
• Configure ECS service autoscaling to spin up instances of your task based on the depth of the SQS queue.