Search code examples
dockermarathondcos

How to config Marathon-lb to load balance a service launched in HOST network?


I have a cluster managed by DC/OS and a dockerized service that I want to deploy through Marathon. I already have a marathon-lb that is being used for service discovery and load balancing of other existing services. All these services are deployed using BRIDGE network.

The new service exposes more than one ports. Port A is for communication between the instances of the service and port B is for accepting requests from the world. I want to use HOST (and not BRIGE) network for deploying the service.

I would like to know how to configure the json of the service in order for marathon-lb to load-balance and expose externally port B.

I have already tried various scenarios and configurations but none worked. The json that I have constructed is the below.

{
  "id": "/cassandra-seed",   
  "cpus": 1.5,
  "mem": 8192,
  "disk": 0,
  "instances": 1,
  "container": {
    "type": "DOCKER",
    "docker": {
      "image": "cassandra:2.2.3",
      "network": "HOST",
      "requirePorts": true,
      "privileged": true,
      "forcePullImage": false     
    }
  },
  "constraints": [["hostname","UNIQUE"]],
  "labels": {
    "HAPROXY_GROUP": "external"
  },
  "portDefinitions": [
    { "port": portA,"protocol": "tcp"},
    { "port": portB,"protocol": "tcp"}
  ]
}

In Marathon documentation it is stated that by explicitly defining port B in portDefitions and setting requirePorts to true, service ports are equal to host port. Also, I deployed a new version of marathon-lb where port B is its portDefinitions section (violating the 10000-10100 default range).

Thus, I assumed that by providing the HAPROXY_GROUP label in the service json, marathon-lb would expose port B as desired. However, this does not seem to be the case. If I deploy the service and curl http://marathon-lb.marathon.mesos:portB the response is "Empty reply from server". However, If I curl http://physicalNodeIP:portB I can connect to an instance of the service.

Thank you in advance.


Solution

  • It looks like you have requirePorts in the wrong section of the app definition. It should be at the top level, like this:

    {
      "id": "/cassandra-seed",   
      "cpus": 1.5,
      "mem": 8192,
      "disk": 0,
      "instances": 1,
      "container": {
        "type": "DOCKER",
        "docker": {
          "image": "cassandra:2.2.3",
          "network": "HOST",
          "privileged": true,
          "forcePullImage": false     
        }
      },
      "constraints": [["hostname","UNIQUE"]],
      "labels": {
        "HAPROXY_GROUP": "external"
      },
      "requirePorts": true,
      "portDefinitions": [
        { "port": portA,"protocol": "tcp"},
        { "port": portB,"protocol": "tcp"}
      ]
    }
    

    As a sidenote, you should consider using the Cassandra framework, as opposed to running Cassandra on Marathon.