Search code examples
dockermaster-slavemesosdocker-composemarathon

Configure 3 Mesos instance with 1 master using docker and docker-compose


By reading this article : how-to-configure-a-production-ready-mesosphere-cluster-on-ubuntu-14-04, I wanted to start my own docker mesosphere using 3 server.

The setup is similar than the article, expect I use 4 dockerized server :

  • Docker Zookeeper
  • Docker Mesos Master
  • Docker Mesos Slave
  • Docker Marathon

I got really confused by the configuration files location, because they install the 4 components on the same machine. Docker install use 4 different server, how do you apply the steps correctly using Docker.

I have

  • Server 1 - prod02 - prod02.domain.com
  • Server 2 - preprod02 - preprod02.domain.com
  • Server 3 - prod01 - prod01.domain.com

Here is a the docker-compose.yml I started writting for running the master mesosphere server 1

zookeeper:
  build: zookeeper
  restart: always
  command: /usr/share/zookeeper/bin/zkServer.sh start-foreground
  ports:
    - "2181:2181"
    - "2888:2888"
    - "3888:3888"
master:
  build: master
  restart: always
  environment:
    - MESOS_HOSTNAME=master.prod-02.example.com
    - MESOS_ZK=zk://prod-02.example.com:2181,prod-01.example.com:2181,preprod-02.example.com:2181/mesos
    - MESOS_QUORUM=1
    - MESOS_LOG_DIR=/var/log/mesos
    - MESOS_WORK_DIR=/var/lib/mesos
  volumes:
    - /srv/docker/mesos-master:/var/log/mesos
  ports:
    - "5050:5050"
slave:
  build: slave
  restart: always
  privileged: true
  environment:
    - MESOS_HOSTNAME=slave.prod-02.example.com
    - MESOS_MASTER=zk://prod-02.example.com:2181,prod-01.example.com:2181,preprod-02.example.com:2181/mesos
    - MESOS_EXECUTOR_REGISTRATION_TIMEOUT=5mins #also in Dockerfile
    - MESOS_CONTAINERIZERS=docker,mesos
    - MESOS_LOG_DIR=/var/log/mesos
    - MESOS_LOGGING_LEVEL=INFO
  volumes:
    - /var/run/docker.sock:/var/run/docker.sock
    - /usr/bin/docker:/usr/bin/docker
    - /sys:/sys:ro
    - /srv/docker/mesos-slave:/var/log/mesos
    - /srv/docker/mesos-data/docker.tar.gz:/etc/docker.tar.gz
  ports:
    - "5051:5051"
marathon:
  build: marathon
  restart: always
  environment:
    - MARATHON_HOSTNAME=marathon.prod-02.example.com
    - MARATHON_MASTER=zk://prod-02.example.com:2181,prod-01.example.com:2181,preprod-02.example.com:2181/mesos
    - MARATHON_ZK=zk://prod-02.example.com:2181,prod-01.example.com:2181,preprod-02.example.com:2181/marathon
  ports:
    - "8081:8080"

My project directory looks like this

  • /prod-02
    • /marathon
      • Dockerfile
    • /master
      • Dockerfile
    • /slave
      • Dockerfile
    • /zookeeper
      • /assets
      • /conf
        • myid
        • zoo.cfg
    • docker-compose.yml

With this config, the master and slave serveur can't start , log is :

WARNING: Logging before InitGoogleLogging() is written to STDERR
F1016 12:12:49.976361     1 process.cpp:895] Failed to initialize: Failed to bind on XXX.XXX.XXX.XXX:5051: Cannot assign requested address: Cannot assign requested address [99]
*** Check failure stack trace: ***

I feel a bit lost due to lake of documentation, any help to configure is well appreciated


Solution

  • I finally sort this out, what was missing was the external ip address MESOS_IP set for master and slave and also the net: host mode