Our team does a ton of scheduled automation work that we would like to move to the cloud. Much of it is already dockerized, which makes moving it to ECS trivial.
What I Want
What I want to do is create a Docker image, push it to ECR, and have that image run as a container every 15 minutes and exit when complete. All the container needs to do is run the script to completion (takes only a minute or so) and shut down.
What Is Happening
The synth and deploy work. The repository and image are created, as are the cluster, service, task, and schedule. Unfortunately, however, my container is spun up every minute instead of every fifteen in the schedule I have created. [edit] I can see in the logs that the container ran successfully.
What I Suspect
I suspect that despite everything I have tried, AWS doesn't know the health state of the container when it exits, so it retries it. I am operating under the assumption that the task should end when the container exits, but maybe I am wrong or missing something.
What I Have Tried
So far I have tried setting retries to 0 in my stack definition, and forcing my script to exit with sys.exit(0) when complete, but nothing seems to work. I have tried both cron and duration and neither setting has changed the outcome.
Other Options
I am betting the advice is going to be adding a container-backed lambda and running that instead of the container directly. I'd like to avoid the additional complexity if possible, but if I have to go that route, I can.
I'd appreciate any comments or suggestions to get around this issue!
Regards, Bruber
For what you're describing, a single-node job in AWS Batch sounds like a better fit. As long as your job exits with 0, it'll trigger success for the job.
AWS Batch leverages ECS to run your containers and simplifies the orchestration.