Search code examples
amazon-web-serviceskubernetesmesosmesosphere

Expose Kubernete service to public inside mesosphere's DCOS

Followed https://www.mesosphere.com/amazon/ I created a DCOS cluster on Amazon AWS. Then I followed http://kubernetes.io/v1.1/docs/getting-started-guides/dcos.html and installed Kubernete on it. Then I followed http://kubernetes.io/v1.1/docs/user-guide/quick-start.html I was able to launch pods successfully. Then I ran into problem with expose the service to public.

$ dcos kubectl expose rc my-nginx --port=80 --type=LoadBalancer
service "my-nginx" exposed
$ dcos ssun$ dcos kubectl get svc my-nginx
NAME       CLUSTER_IP    EXTERNAL_IP   PORT(S)   SELECTOR       AGE
my-nginx   10.10.10.32                 80/TCP    run=my-nginx   8s

The EXTERNAL_IP address does not exists. According to the tutorial, it should. So I'm thinking it has something to do with the fact that my Kubernete is inside DCOS.

Please help. Thank you very much!

Solution

  • Kubernetes on Mesos/DCOS does not support automatic LoadBalancer creation yet.

    As the quick start states:

    Through integration with some cloud providers (for example Google Compute Engine and AWS EC2), Kubernetes enables you to request that it provision a public IP address for your application.

    AFAIK, only GCE, GKE, and AWS support automatic LoadBalancer creation so far.

    Another key difference about DCOS (compared to kubernetes) is that it comes by default with two zones: public and private. So nothing scheduled on the private nodes is externally accessible without a reverse-proxy on the public nodes.

    Additionally, Kubernetes on DCOS does not yet support IP-per-container. Support for IP-per-container is under development with the DCOS/Calico integration. Some community members have also reportedly attempted using cluster-wide docker overlay networking.

    For now, there are a few alternative options for reaching your pod externally:

    1. Deploy your pod on all the public slaves (using resource role annotations) and hostPort:80. Then hit the address of the DCOS public slave AWS ELB.
    2. Create your own load balancer nginx pod (e.g. service-loadbalancer and schedule it on the public slaves with hostPort:80. Then hit the IP of the host node it's on.

    It's definitely a priority of the Mesosphere Kubernetes Team to make this experience smoother on DCOS. Hopefully the solution will include automatic LoadBalancer creation.