Search code examples
chronicle-map

ChronicleMap Recovery with multi process application

We are evaluating ChronicleMap and our application runs cluster mode with nodes ranging from 5 to 45. The plan is to have the ChronicleMap persisted in shared NFS folder so that all the nodes can read/write.

There are more likely chance that individual nodes could go down for various reasons in the middle of a read/write operation with this said. I have some questions

  1. If node-1 goes down during a write operation, can another healthy node-2 in the cluster still continue to read/write to the files?
  2. Lets say we implement some logic to detect a server crash and call the .recoverPersistedTo() on restart. Will this cause any issues while other healthy nodes in the cluster are reading/writing to the files? The reason I ask this question is that the document says

“You must ensure that no other process is accessing the Chronicle Map store when calling .recoverPersistedTo()”

  1. I have read that using .recoverPersistedTo() in place is createPersistedTo() is not a good practice, but what are the downsides?
Solution

  • First of all, we (Chronicle) don't support putting Chronicle Map files on NFS (as we use memory mapping and NFS is known to cause problems with it). Additionally, trying to use recovery on NFS will cause data corruption as there's no adequate file locking on NFS, and recovery tries to lock the file to prevent simultaneous recovery by multiple processes. In general, open source Chronicle Map is supposed to be used by multiple processes on the same host.

    The solution to your problem is commercial Map Enterprise which supports map replication between nodes, please contact [email protected] for details.