Search code examples
dockerdocker-swarm

Should I constrain swarm services to worker nodes?


Say I have a docker swarm with separate workers and managers. Do I have to specify a placement constraint to ensure that a given service only runs on a worker node?

Like in this docker-compose file:

version "3.1"

services: 
  foo:
    image: hello-world
    deploy:
      placement:
         - node.role==worker

Or does Swarm automatically limit services to run on worker nodes?


Solution

  • If you come from Kubernetes, there's a concept of tainting a node which is done for each of the managers, and then to run a workload on a manager, you need to configure that pod to run on a node with that taint. That doesn't exactly exist in Swarm Mode. But you have a couple of options to implement it yourself.

    The most common method is as you've described, make a placement constraint on node.role==worker for every service you only want to run on workers vs the manager. This requires a bit of work on every service that you plan on scheduling on the cluster, but it is also one of the more flexible options since you can specify some workloads do need to run on managers (e.g. any utility service that needs to manage the cluster or have access to the manager details of the cluster, like self configuring reverse proxies).

    The next method is to change the availability of the managers:

    $ docker node update --help
    Usage:  docker node update [OPTIONS] NODE
    Options:
          --availability string   Availability of the node ("active"|"pause"|"drain")
    ...
    

    By default, all nodes are active. However, you can pause new workloads from being scheduled on a node, or drain existing workloads off of a node, by changing its availability. This does not impact containers being deployed outside of swarm mode (with docker container run or docker-compose instead of Swarm Mode's docker stack or docker service commands). However, it's inflexible and doesn't allow you to run some swarm containers on managers and others on workers easily. The most common use of changing the availability setting is to take a node out of service for maintenance rather than to control scheduling on managers. The only other time I've used this is when a third manager was added to a two node cluster for HA where it should never have any workload scheduled on it.

    A third method to implement this is part of Docker's EE offering. When you schedule jobs using Docker UCP, it acts as a security layer between the user running commands and the docker nodes. One of the features of UCP is controlling what nodes users can schedule their workload on, and there's a top level switch to enable or disable scheduling jobs on managers by users and even by administrators. Even if you didn't go with Docker EE, it is possible to write your own docker API proxy that users would have access to, and that adds additional scheduling constraints on to services being deployed. But given the ease of the first option, I've yet to see anyone do this.