High Latency between Hazelcast members (using Topic publish/onMessage)

Question

I'm using Hazelcast (v3.5.4) Topics for a really simple chat application running on two Azure VMs (Standard D3).

When publishing messages sometimes it takes up to 15 seconds to be received by the other member. I logged publish and onMessage methods to be clear that hazelcast is causing the delay. There is no network delay and the applications and VMs are using nearly zero CPU resources. What can cause such a delay?

For messaging Topics are used:

private ITopic<ChatMessage> eventTopic;

@PostConstruct
private void init() {
    eventTopic = hazelcastInstance.getTopic("chat-messages");
    eventTopic.addMessageListener(new ChatMessageListener());
}

public void publishMessage(final ChatMessage chatMessage) {
    log.debug("Publish message: " + chatMessage.toString());
    eventTopic.publish(chatMessage);
}

The two azure VMs are in the same region and the latency (Ping) between these two VMs is under 5ms.

Multicast join is disabled, static TcpJoin is used:

@Bean
public HazelcastInstance hazelcastInstance() {
    final Config config = new Config();
    NetworkConfig networkConfig = config.getNetworkConfig();

    networkConfig.setPort(5701);
    networkConfig.setPortAutoIncrement(true);
    networkConfig.setPortCount(3);
    networkConfig.getInterfaces()
        .addInterface("10.0.0.*")
        .setEnabled(true);
    final JoinConfig join = networkConfig.getJoin();
    join.getAwsConfig().setEnabled(false);
    join.getMulticastConfig().setEnabled(false);
    join.getTcpIpConfig().addMember("10.0.0.1-2").setEnabled(true);

    return HazelcastInstanceFactory.newHazelcastInstance(config);
}

Spring Boot 1.3 is used with hazelcast-spring 3.5.4

Answer 1

Thanks for you replies, I found the cause of the delay, in my MessageListener I used

message.getPublishingMember().getSocketAddress().getHostName()

for logging. On windows machines the getHostName method sometimes takes up to 15 seconds.