How to use Hazelcast's CPSubsystem with fewer than 3 nodes?

Question

I see that Hazelcast 3.12 has introduced the CPSubsystem() for systems with 3-7 nodes. I understand the reasoning. However, if I am trying to design a solution that can run with anywhere between 1-n nodes, do I need to use different logic to validate if the CPSubsystem is enabled? How do I even check that?

I would have thought/hoped that simply calling

hazelcastInstance.getCPSubsystem().getLock()

would work no matter the number of nodes, but if there are fewer than 3 nodes, it throws an exception. And I can't find any method that allows me to check if the CPSubsystem is enabled or not.

My current implementation uses the deprecated method getLock() to get a distributed lock:

   LOG.debug("Creating a distributed lock on username for a maximum of 5 minutes {}", username);
    ILock usernameLock = hazelcastInstance.getLock(this.getClass().getName() + ":" + username);
    try {
        if (usernameLock.tryLock (5, TimeUnit.MINUTES)) {
            clearUserData(cacheEntryEvent);
        }
    } catch (InterruptedException e) {
        LOG.warn("Exception locking on : {} ", username, e);
        LOG.warn("Invoking clearUserData without synchronization : {}", username);
        clearUserData(cacheEntryEvent);
    } finally {
        usernameLock.unlock();
    }

How can I get a lock with Hazelcast without knowing this? The hazelcastInstance.getLock() is marked as deprecated and targeted for removal in HC4.

Answer 1

As you already know, CPSubsystem is a CP system in terms of CAP theorem. It has to be enabled explicitly to use, because it has some limitations and prerequisites. One of them is, at least 3 Hazelcast members should exist in the cluster. Actually, 2 members is sufficient but Hazelcast's CPSubsystem rejects to work with 2 members because majority of 2 members is 2 again, and it's prone to be unavailable once one of the members crashes.

HazelcastInstance.getLock() uses async replication of Hazelcast and cannot provide CP guarantees under failures. This is fine for some systems/applications but not for all. That's why choosing between a best-effort locking mechanism vs CP based locking mechanism should be explicit and applications relying on the lock should be designed depending on this choice. See Daniel Abadi's The dangers of conditional consistency guarantees post related to this choice. That's why, CPSubsystem().getLock() does not fallback to best-effort/unsafe locking mechanism when cluster size is below 3.

HazelcastInstance.getLock() is deprecated in 3.12 and will be removed in 4.0. But Hazelcast will provide an unsafe (development) mode for CP data structures, which will work with any number of members and will be based on async replication similar to Hazelcast AP data structures.