Search code examples
amazon-ec2google-compute-enginegoogle-kubernetes-engineamazon-ecs

Low-cost daily cloud container job execution


I have a couple of jobs which I want to execute on a cloud computing platform which need approx 30 minutes daily.

I do not want to pay for compute resources the remaining 23,5h.

What options for compute cloud platforms are there which allow me to run these jobs at very low cost?

Using Amazon EC2 Container Service, I do not constantly want have a cluster with running instances when I do not need them. I can destroy and create the cluster daily, but this seems inefficient. Can I use ASG to scale down to zero running instances?

Google Compute Cloud seems to allow me to keep a cluster configured, but scale down the number of instances in a container cluster to zero pretty effortlessly, and then scale up again. But how to trigger this scaling automatically once tasks are done?

If I use Amazon EC2, can I something like AWS Lambda functions for this?

Is there some container-centric service where I only pay for container execution time and not for compute resources?


Solution

  • There are a lot of incorrect terms here.

    You can use an auto-scaling group with timed scales to spin up/down daily. You can then run the task.

    If you'd like to fully automate it I would instead use a Lambda to spin up the ECS cluster, wait for it to be ready, then run the tasks and kill the instances.

    You would be billed for the lambda execution time, the time the instances were up and any other services you used in the mean time.

    If you wanted it to be quicker you could create the ECS cluster and join the instances to it, but then shut them off. You incur no billing while they are off. Then your lambda could start them, wait until the ECS cluster is ready, then execute.

    Lambda and Autoscaling groups have timer triggers. AWS charges you for the infrastructure, they are not PaaS, meaning you can't just run a task and pay for that, you pay for the infrastructure and do the execution yourself. Lambda is the only service mentioned here that requires no hosting and charges for the "execution time".