Search code examples
network-programmingamazon-ec2architecturemicroservicesconsul

Consul in a Docker-based Microservices architecture

We are working on switching over to micro services from a monolithic application. Each microservice is going to be running on Docker through Amazon ECS.

We've decided to use Consul for service discovery. We have 3 servers running on EC2 instances inside the VPC.

My question is as follows:

How/Where do I start the Consul agent for each micro service? Do I run another container on each instance (through Docker-Compose) with Consul inside? Or do I somehow run a Consul agent inside the already existing Docker container for each micro service?

enter image description here

Attached is a rough representation of my situation. Should the Consul Client (in yellow) be in its own Docker Container or inside the Node.js container?

Solution

  • Consul is another service, and I wouldn't deploy it inside the container of my microservice. In a large-scale scenario, I'd deploy several Consul containers: some would run the agent under Server mode (think of them as Masters), and some would run it under Client mode (think of them as Slaves).

    I wouldn't deploy the agents running under client mode as part of my application's containers, because:

    1. Isolating them means that they are stopped individually. Putting them together means that whenever I'd stop my application's container due to a version upgrade or a failure, I'd be needlessly stopping the Consul agent running therein. Same goes the other way around: stopping the Consul agent would stop my running application. This unneeded coupling isn't beneficial.
    2. Isolating them means they can be scaled separately. I may need to scale my microservice and deploy more instances of it. If the container also contains Consul client agents, then scaling my microservice would end up scaling Consul as well. Or the other way around: I may need to scale Consul without scaling my microservice.
    3. Isolating them is easier in terms of Docker container images. I can keep using the official Consul image and upgrade without much hassle. Putting Consul and my microservice together would mean that upgrading Consul requires me to modify the container image by myself.