Search code examples
amazon-web-servicesdockerdocker-composeterraformamazon-ecs

Terraform + AWS ECS, continuous deployment process?

I'm currently migrating to AWS ECS, after using AWS Elastic Beanstalk with the Ruby platform to host & deploy my Rails app for the last 6+ years.

As such, I've had to dockerize my app and other services with docker-compose, and am now considering how to provision the infrastructure and prepare code deployments.

I've spent weeks studying to make this migration, but I'm now a bit overwhelmed with the variety of methods to deploy the infrastructure and app to AWS (co-pilot, docker-compose, eb w/ docker, etc.).

Also, I was hoping to remain as cloud-platform-agnostic as possible, as I will be moving some of my AWS infrastructure to Digital Ocean in the future. So I was hoping to use Terraform, but since the options mentioned above all seem to use CloudFormation internally, Terraform seems unnecessary.

Given that context, what processes and tools should I be using if I want to:

  1. Provision my infrastructure on AWS with Terraform.

  2. Quickly and simply deploy code changes to production with no downtime. Ideally, we wouldn't use AWS-based Dev-ops tools (CodeCommit, CodeDeploy, etc).

I'm new to docker and elastic beanstalk made code deployments so easy, that I'm feeling uncertain about where else I should be looking. I've never used tools like Jenkins before and am not entirely sure if/how that fits into my processes.

Thank you for the guidance!

Solution

  • Continuous integration/continuous deployment

    A tools whose main responsibility is CI/CD (e.g. CodePipeline, Jenkins, CircleCI, GitHub Actions) is the piece of the puzzle that will tie everything else together. This will allow you to clearly define sequential steps for your deployment pipeline.

    Defining those individual steps first requires answering some questions (non-exhaustive!):

    • Are you going to build your Docker image once per environment or just once in total?
    • Are you building and publishing your Docker image to an artifact registry (Docker Hub or AWS ECR, e.g.) before later pulling it in your deployment step?
    • Do you want every push/merge to/into main/master to trigger your pipeline or do you want to manually trigger some/all of it?
    • Do you need to use Docker Compose or will Docker be sufficient? What other services are you running?
    • How do you build and run your app locally?

    Implementation

    The likely scenario is that you will end up using the Docker, Terraform and AWS CLIs in some sort of Linux container shell to actually achieve your goals. Most modern CI/CD-as-a-service providers allow you to choose a Docker image for each stage of your pipeline(s), so you should be able to select an image that already meets your needs.

    Separating infrastructure provisioning and app deployment

    This can be done within the same Git project but by using separate pipelines/jobs/stages (name depends on your CI/CD provider of choice). Alternatively, you may opt for a separate repo. Personally, I find that keeping them together makes it easier to keep things in sync.

    You should also be clear about where Terraform's responsibilities end. While it is possible to use Terraform to deploy a container, I would argue that it should be confined to the underlying infrastructure.

    Staying cloud-agnostic

    Choosing something like GitHub Actions or CircleCI will give you the option of multi-cloud deployments (e.g.) in the future. However, if it comes down to opting for something like CodeDeploy over writing a lot of shells scripts to get the deployment behaviour you desire, then I can see it being worth incurring the future migration costs for the time and effort saved in the present.