Search code examples
dockerdocker-swarmdocker-ingress

Docker Swarm Mode violates the constraint at creation?

I am currently trying to deploy a psql cluster in a docker swarm, therefore I have created a swarm with currently 2 nodes. (psql_manager, psql_worker1)

My plan was to have a primary node which i could use from outside and secondarys which will be used to serve none write querys. My only problem exists in the way the docker swarm creates services.

My current commands so far (manager):

docker service create \
--mount type=volume,src=pg-primary-volume,dst=/pgdata,volume-driver=local \
--name primary \
--constraint 'node.role==manager' \
--env PGHOST=/tmp \
--env PG_MODE=primary \
--env PG_USER=user \
--env PG_PASSWORD=passwort  \
--env PG_DATABASE=test \
--env PG_PRIMARY_USER=tester \
--env PG_PRIMARY_PASSWORD=passwort  \
--env PG_PRIMARY_PORT=5432 \
--env PG_ROOT_PASSWORD=passwort \
--publish published=6432,target=6432 \
--publish published=5433,target=5433 \
--publish published=5432,target=5432 \
crunchydata/crunchy-postgres:centos7-10.9-2.4.1

And the replica:

docker service create \
--mount type=volume,src=pg-primary-volume,dst=/pgdata,volume-driver=local \
--constraint 'node.role!=manager' \
--env PGHOST=/tmp \
--env PG_MODE=replica \
--env PG_USER=user \
--env PG_PASSWORD=passwort  \
--env PG_DATABASE=test \
--env PG_PRIMARY_HOST=psql_ip \
--env PG_PRIMARY_USER=tester \
--env PG_PRIMARY_PASSWORD=passwort  \
--env PG_PRIMARY_PORT=5432 \
--env PG_ROOT_PASSWORD=passwort \
--publish published=6432,target=6432 \
--publish published=5433,target=5433 \
--publish published=5432,target=5432 \
--mode global --detach=true crunchydata/crunchy-postgres:centos7-10.9-2.4.1

The expected result is that there is one service created: primary which can work right away. And the worker will get the replica and will work then too (wrks over the global arg). No port collision or anything, because it would violate the constrain check. But i get the collision anyway

port '6432' is already in use by service 'primary' (xunlw3nsdyvrveg5vo4it40ag) as an ingress port

There is a option to enable the publish method host but that would turn off the ingress mesh routing and i want to use it (btw then it works fine no collisions).

Why does the constraint not work like intented on the swarm with ingress mesh enabled?

Solution

  • ingress / swarms mesh routing is very all or nothing. Once one service owns a port, no other service can own the port.

    So, either you need to use mode: host and somehow loadbalance between docker nodes...

    or, I would be very tempted to replace the entrypoint of the image with a custom entrypoint shell script. and then deploy a single service with

      entrypoint: ["my-entrypoint.sh"]
  configs:
    - source: my-postgres-entrypoint
      target: /usr/local/bin/my-entrypoint.sh
      mode: 0755
  deploy:
    replicas: 2
  environment:
    NODE_HOSTNAME: "{{.Node.Hostname}}"

    And then your entrypoint script will switch behavior if $NODE_HOSTNAME is your manager node.