Search code examples
amazon-web-servicesamazon-ecsaws-code-deployaws-fargate

How to best deploy code changes to ECS using CodeDeploy?


We deploy a Docker image that runs a simple Sinatra API to an ECS Fargate service. Right now our task definition defines the image using a :production tag. We want to use CodeDeploy for a blue/green deployment.

When code is changed - should we push a new image with the :production tag and force a new deployment on our service or instead use specific tags in our task definition (e.g. :97b9d390d869874c35c325632af0fc1c08e013cd) and create a new task revision then update our service to use this new task revision?

Our concern with the second approach is that we don't see any lifecycle rules around task revisions so will they just build up until we have tens/hundreds of thousands?

If we use the first approach, will CodeDeploy be able to roll back a failed deployment in the case there is an issue?


Solution

  • Short answer

    In both cases there are no definition roll back if somehow your new image crashed but your current old task should still be alive. But if you are using health check and the current running task is below the required amount (might be due to overflow of user traffic,...etc), Fargate would start up new task with the latest task definition revision which contained the bad image.

    Long answer

    Since you are just asking CodeDeploy to start up task based on your image, it would create a new task definition that have your image's URI to pull the correct image. And that new task definition would always be used to start up new Fargate task.

    So when Fargate found that it needs to create task, it would always try to use the latest revision which would always be the one with bad image.

    The good thing is that your old image task if works correctly, it should still be alive, since the minimum running task would be 1 and since the other task is failing continuously, your old image task would not be decommissioned.

    You can however overcome this by adding a CloudWatch event to trigger a lambda that either update new task revision with the good image tag or running current Fargate with the previous task definition revision. Here is an article from AWS about this: https://aws.amazon.com/blogs/compute/automating-rollback-of-failed-amazon-ecs-deployments/

    A bit more on how Fargate deployment work here and help your old task running when new deployment fail, it would first provision the new task, when all the new tasks is running good, it would decommission old task. So in case the new tasks does not run properly, old task should still be alive.