I am running docker swarm with consul. I have a consul cluster made with 3 nodes connected to each other for failover. The issue is that I can only connect my swarm workers and masters to a single node and if that node goes down then swarm stops working. So how can I connect swarm workers and masters to all my nodes? The following commands if run from the master will set up my swarm environment connected to a single consul server:
#### REFERENCE
# {{master_i}} is the IP address of the master server
# {{consul_i}} is the IP address of the consul server
# {{worker_i}} is the IP address of a worker server
#### START THE MASTER
docker run --restart=unless-stopped --name=swarm-manager0 -d -p 4000:4000 swarm manage -H :4000 --replication \
--advertise {{master_0}}:4000 \
consul://{{consul_0}}:8500
#### START THE WORKERS REMOTELY FROM THE MASTER
docker -H={{worker_0}}:2375 run -d --restart=unless-stopped --name=swarm-worker0 swarm join \
--advertise={{worker_0}}:2375 \
consul://{{consul_0}}:8500/
docker -H={{worker_1}}:2375 run -d --restart=unless-stopped --name=swarm-worker1 swarm join \
--advertise={{worker_1}}:2375 \
consul://{{consul_0}}:8500/
docker -H={{worker_2}}:2375 run -d --restart=unless-stopped --name=swarm-worker2 swarm join \
--advertise={{worker_2}}:2375 \
consul://{{consul_0}}:8500/
#### START THE WORKER SERVICE DISCOVERY
docker -H={{worker_0}}:2375 run -d --restart=unless-stopped \
-h {{worker_0}} --name registrator0 -v /var/run/docker.sock:/tmp/docker.sock gliderlabs/registrator \
consul://{{consul_0}}:8500
docker -H={{worker_1}}:2375 run -d --restart=unless-stopped \
-h {{worker_1}} --name registrator1 -v /var/run/docker.sock:/tmp/docker.sock gliderlabs/registrator \
consul://{{consul_0}}:8500
docker -H={{worker_2}}:2375 run -d --restart=unless-stopped \
-h {{worker_2}} --name registrator2 -v /var/run/docker.sock:/tmp/docker.sock gliderlabs/registrator \
consul://{{consul_0}}:8500
Note that simply adding two extra consul://{{consul_i}}:8500
(for the other two consul servers) to the end of each docker run command will not connect the containers to the other consul servers.
According to @slugonamission there is no way to connect swarm to the multiple IP addresses of multiple consul servers.
However I was able to create an haproxy load balancer that sat in front of my consul servers. So my load balancer forwarded all traffic from my load balancers port 8500 on to port 8500 on all of my consul servers. By doing this I was able to use the IP address of my load balancer in place of {{CONSUL0}}
. Heres my pretty basic haproxy.cfg
# $CONSUL0 $CONSUL0 and $CONSUL0 are the IP addresses of my consul servers
global
log 127.0.0.1 local0 notice
maxconn 2000
user haproxy
group haproxy
defaults
log global
mode http
option httplog
option dontlognull
retries 3
option redispatch
timeout connect 5000
timeout client 10000
timeout server 10000
listen appname 0.0.0.0:8500
mode http
stats enable
stats uri /haproxy?stats
stats realm Strictly\ Private
stats auth ubuntu
balance roundrobin
option httpclose
option forwardfor
server consul0 $CONSUL0:8500 check
server consul1 $CONSUL1:8500 check
server consul2 $CONSUL2:8500 check
After making the change my consul servers can individually go down and swarm will continue working.