Search code examples
dockerdocker-composedockerfiledocker-swarm

Docker-Compose scale and persistant data storage


Currently, I'm working on a project implementing a custom docker image with a postfix mail server. Docker will provide us the ability to scale the mail service up/down.

The question I have is regarding data storage. Within each container instance, I would like to write the contents of /var/log to the host or even a remote host if possible. As expected, with my current docker-compose file scaled instances are all using the same storage location.

What I'm looking to find is if something like dynamic storage exists.

For example, If I scale up to 3 instances, then I would want to see 3 subdirectories under /var/lib/docker/volumes/MYVOLUME/

Below is my docker-compose.yml file for reference.

version: '3.2'
services:
  sd-corp:
    build: ./Corporate
    tty: true
    networks:
      corpnet:
    volumes:
      - type: volume
        source: corp
        target: /var/log
        volume:
          nocopy: true
  sd-ent:
    build: ./Entertainment
    tty: true
    networks:
      entnet:
    volumes:
      - type: volume
        source: ent
        target: /var/log
        volume:
          nocopy: true
  sd-soft:
    build: ./Software
    tty: true
    networks:
      softnet:
    volumes:
      - type: volume
        source: soft
        target: /var/log
        volume:
          nocopy: true
networks:
  corpnet:
    driver: bridge
    ipam:
      config:
      - subnet: 10.9.50.0/24
  entnet:
    driver: bridge
    ipam:
      config:
      - subnet: 10.9.51.0/24
  softnet:
    driver: bridge
    ipam:
      config:
      - subnet: 10.9.52.0/24
volumes:
  corp:
  ent:
  soft:

Solution

  • Are these just standard system log files? They shouldn't log to file, they should lot to STDOUT and let docker handle them with a logging driver, which is a much easier way to deal with container logs.

    Here's how the official Nginx image does it, by symlinking logs to stdout/stderr.