Docker containers on same host listening to the same 2 ports

Question

I am trying to make different Python scripts running on different containers , configured with docker compose able to listen to the same ports. Each of this containers (that are 9) should listen to port 1883 and 8086. The idea is to write a docker-compose file in which each service have these characteristics:

  sensor1:
    build: ./sensor1
    image: sensor1:latest
    ports:
     - "8086:80"
     - "1883:80"
....
....
  sensor9:
    build: ./sensor9
    image: sensor9:latest
    ports:
     - "8086:80"
     - "1883:80"

I know that with a normal docker compose file it doesn't work and I need a reverse proxy, but I am stuck on this reverse proxy(i.e. Traefik) configuration. Practically these scripts should listen to port 1883 receiving from a external broker and write on a database positioned in the same host that can be accessed using port 8086

Answer 1

Binding to multiple addresses

You cannot bind ports from multiple containers to the same host ports when listening on the same host address. The only way to make a configuration like that work is to bind the ports to different addresses on the host. For example, if I have multiple addresses associated with eth0 on my host:

$ ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 192.168.1.175/24 brd 192.168.1.255 scope global dynamic noprefixroute eth0
       valid_lft 49625sec preferred_lft 49625sec
    inet 192.168.1.200/24 scope global secondary eth0
       valid_lft forever preferred_lft forever
    inet 192.168.1.201/24 scope global secondary eth0
       valid_lft forever preferred_lft forever

Then I can bind each of my containers to a specific address, like this:

  sensor1:
    build: ./sensor1
    image: sensor1:latest
    ports:
     - "192.168.1.175:8086:80"
     - "192.168.1.175:1883:80"
[...]
  sensor2:
    build: ./sensor2
    image: sensor2:latest
    ports:
     - "192.168.1.200:8086:80"
     - "192.168.1.200:1883:80"
[...]

Then a connection to http://192.168.1.175:8086 will go to the sensor1 container, while a connection to http://192.168.1.200:8086 would go to the sensor2 container.

Hostname and path based routing

If you want everything hosted at the same address, then you need another strategy for differentiating between the containers. Your options are effectively:

• Hostname -- you configure multiple hostnames to point to the same ip address, and a load balancer like Traefik will use the hostname to direct incoming connections to the appropriate container.

• Path -- each container is exposed at a different path (e.g., http://myhost/sensor1 goes to sensor1, http://myhost/sensor2 goes to sensor2, etc). The load balancer uses the path contained in incoming requests to route traffic.

Path example

I'll start with the path example, because that's often easiest. It doesn't require setting up DNS entries or mucking about with /etc/hosts on multiple machines.

The following docker-compose.yaml demonstrates a path-based routing configuration:

version: '3'

services:

  # This is the load balancer. To match the configuration you show in
  # your question, I have it listening on ports 8086 and 1883 in
  # addition to port 80.
  #
  # The default configuration of Traefik is to expose a management
  # interface on port 8080; if you don't want that, you can remove
  # the corresponding `ports` entry.
  reverse-proxy:
    image: traefik:v2.7
    command: --api.insecure=true --providers.docker
    ports:
      - "80:80"
      - "8086:80"
      - "1883:80"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

  # In our container configuration, we use labels to configure
  # Traefik. Here, we're declaring that requests prefixed by `/sensor1`
  # will be routed to this container, and then we strip the `/sensor1`
  # prefix from the request (so that the service running inside the
  # container doesn't see the prefix).
  #
  # Note that we're not publishing any ports here: only the load
  # balancer has ports published on the host.
  sensor1:
    hostname: sensor1
    labels:
      - traefik.enable=true
      - traefik.http.routers.sensor1.rule=PathPrefix(`/sensor1`)
      - traefik.http.services.sensor1.loadbalancer.server.port=80
      - traefik.http.middlewares.strip-sensor1.stripprefix.prefixes=/sensor1
      - traefik.http.routers.sensor1.middlewares=strip-sensor1
    build: ./sensor1

  sensor2:
    hostname: sensor2
    labels:
      - traefik.enable=true
      - traefik.http.routers.sensor2.rule=PathPrefix(`/sensor2`)
      - traefik.http.services.sensor2.loadbalancer.server.port=80
      - traefik.http.middlewares.strip-sensor2.stripprefix.prefixes=/sensor2
      - traefik.http.routers.sensor2.middlewares=strip-sensor2
    build: ./sensor2

If each container is running a service that includes the hostname in /hostname.txt, I will see the following behavior:

$ curl myhost/sensor1/hostname.txt
sensor1
$ curl myhost/sensor2/hostname.txt
sensor2

Hostname example

A host-based configuration looks pretty much identical, except the rule uses a Host match instead of a PathPrefix match (and we no longer need the prefix-stripping logic):

version: '3'

services:
  reverse-proxy:
    image: traefik:v2.7
    command: --api.insecure=true --providers.docker
    ports:
      - "80:80"
      - "8086:80"
      - "1883:80"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

  sensor1:
    hostname: sensor1
    labels:
      - traefik.enable=true
      - traefik.http.routers.sensor1.rule=Host(`sensor1`)
      - traefik.http.services.sensor1.loadbalancer.server.port=8080
    build:
      context: web

  sensor2:
    hostname: sensor2
    labels:
      - traefik.enable=true
      - traefik.http.routers.sensor2.rule=Host(`sensor2`)
      - traefik.http.services.sensor2.loadbalancer.server.port=8080
    build:
      context: web

  sensor3:
    hostname: sensor3
    labels:
      - traefik.enable=true
      - traefik.http.routers.sensor3.rule=Host(`sensor3`)
      - traefik.http.services.sensor3.loadbalancer.server.port=8080
    build:
      context: web

For this to work, you need to have the multiple hostnames mapping to the docker host. You can accomplish this by setting up appropriate DNS entries, or by adding an appropriate entry to /etc/hosts on any machines that need to contact these services.

We can demonstrate the configuration by setting an explicit Host header in our requests:

$ curl -H 'Host: sensor1' myhost/hostname.txt
sensor1
$ curl -H 'Host: sensor2' myhost/hostname.txt
sensor2