Search code examples
dockerdocker-composedocker-swarmhazelcast

Simple way to create Hazelcast cluster on Multi-hosts Docker

I have only a short time to work with Docker so i'm not very clear about docker networking. Currently I'm trying to convert a system to use docker for deployment. The system contains multiple Java application servers: API server, Chat server, Web Workbench, Plugin Server...

I already created docker-compose file and other Dockerfiles for each server also. This is the a part of docker-compose.yml

version: '3.7'

services:
  apiserver:
    build:
      context: ./apiserver
      dockerfile: APIServerDockerfile
  image: apiserver
  container_name: apiserver
  ports:
    - "8000:8000"
  tty: true
  depends_on:
    - mariadb
  networks:
    - chat_net

chatserver:
  build:
    context: ./chatserver
    dockerfile: ChatserverDockerfile
  image: chatserver
  container_name: chatserver
  ports:
    - "8088:8088"
  tty: true
  depends_on:
    - mariadb
  networks:
    - chat_net

mariadb:
  image: mariadb/server:10.3
  environment:
  MYSQL_ROOT_PASSWORD: abc12345
  ports:
    - "3306:3306"
  networks:
    - chat_net
  restart: always
  container_name: mariadb10.3
  volumes:
    - ./mariadb:/var/lib/mysql

networks:
  chat_net:
  external: true

chat_net is a bridge network as default. The system needs to be distributed, which means apiserver and chatserver and others can be installed on different physical servers. Apiserver and chatserver shares hazelcast cluster to communicate. Normally each server has a hazelcast.xml with some information like this:

<group>
  <name>CHAT_NET_OVERALL_V2</name>
  <password>abc123!@#</password>
</group>

<network>
   <port auto-increment="true" port-count="100">8701</port>
   <join>
     <multicast enabled="false"></multicast>
     <tcp-ip enabled="true">
          <member>127.0.0.1</member>
          <member>15.23.24.56</member>
     </tcp-ip>
   </join>
</network>

Hazelcast cluster connects using ICP/IP with server addresses. Now using docker hazelcast cluster not created using bridge network. If I use network_mode:host, they can form cluster, but our case requires multi-hosts so it's not a solution. In my understanding, I need an overlay network. What i tried to do is to create an overlay network, but even if I add IP of servers in overlay network, the hazelcast cluster cannot be created, each server starts their standalone hazelcast.

So anyone please tells me which is the simplest way to create hazelcast cluster in multi-hosts enviroment? I knew there is some other services like Swarm or Kubernetes or Weavenet but not sure which one could be the best fit for my case. Thank you very much.

Solution

  • The topic is explained in the following article: Configuring Hazelcast in Non-Orchestrated Docker Environments

    Basically, without third-party tooling you have two options:

    • either use the network mode host in Docker
    • or combine Docker port mapping (-p 5701:5701) with defining public (external) address in Hazelcast configuration (-Dhazelcast.local.publicAddress=$DOCKER_HOST_IP:$DOCKER_HOST_PORT)