Search code examples
amazon-ec2hazelcastvert.x

Hazelcasts tries ports other than specified


I have two EC2 instances forming a Hazelcast cluster.
Hazelcast I use is in the vertx-hazelcast:3.9.1 package, which runs Hazelcast version 3.12.2.
I also use the hazelcast-aws:2.4 plugin.

My cluster.xml is:

<?xml version="1.0" encoding="UTF-8"?>
<!--
  ~ Copyright 2017 Red Hat, Inc.
  ~
  ~ Red Hat licenses this file to you under the Apache License, version 2.0
  ~ (the "License"); you may not use this file except in compliance with the
  ~ License.  You may obtain a copy of the License at:
  ~
  ~ http://www.apache.org/licenses/LICENSE-2.0
  ~
  ~ Unless required by applicable law or agreed to in writing, software
  ~ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
  ~ WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the
  ~ License for the specific language governing permissions and limitations
  ~ under the License.
  -->
<hazelcast
    xmlns="http://www.hazelcast.com/schema/config"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://www.hazelcast.com/schema/config
           http://www.hazelcast.com/schema/config/hazelcast-config-3.12.xsd">
    <network>
        <port port-count="1" auto-increment="false">5701</port>
        <public-address>x.x.x.x</public-address>
        <join>
            <multicast enabled="false"/>
            <aws enabled="true">
                <security-group-name>security-group-name</security-group-name>
            </aws>
        </join>
    </network>
</hazelcast>

Both instances have the same cluster.xml, but with different entries in <public-address></public-address>.

What happens on cluster startup, and what I'd like to avoid, is that Hazelcast tries connecting to instances in the same security group, using ports 5701-5708, even though I thought I had set up just one port.
It writes unnecessarily to the log, which looks like this:

2021-04-27 10:51:28,671 INFO  com.hazelcast.nio.tcp.TcpIpConnector:65 - [x.x.x.x]:5701 [dev] [3.12.2] Connecting to /x.x.x.x:5703, timeout: 10000, bind-any: true
2021-04-27 10:51:28,682 INFO  com.hazelcast.nio.tcp.TcpIpConnector:65 - [x.x.x.x]:5701 [dev] [3.12.2] Could not connect to: /x.x.x.x:5703. Reason: SocketException[Connection refused to address /x.x.x.x:5704]
2021-04-27 10:51:28,717 INFO  com.hazelcast.internal.cluster.impl.DiscoveryJoiner:65 - [x.x.x.x]:5701 [dev] [3.12.2] [x.x.x.x]:5703 is added to the blacklist.
...

It writes the same output for all ports in the said range.

I seem to have done as suggested here.
How do I stop it trying to use ports other than 5701?


Solution

  • In the "port" tag if you want to use a specific port, set auto-increment to false. If it is set to false, the port-count attribute must be ignored, so remove it:

    <port auto-increment="false">5701</port>
    

    Also add the following line just below the previous one:

    <reuse-address>true</reuse-address>
    

    When you shutdown a cluster member, the server socket port will be in the TIME_WAIT state for the next couple of minutes. If you start the member right after shutting it down, you may not be able to bind it to the same port because it is in the TIME_WAIT state. If you set reuse-address to true, the TIME_WAIT state is ignored and you can bind the member to the same port again. Default value is false. If you set this to true, Hazelcast will use the same port when you restart a member right after you shut it down.