Search code examples
node.jswebsocketcrdtautomerge

Automerge synchronisation

Problem

I have implemented a system where the frontend and backend make changes to a CRDT document.

The frontend and backend communicate via websockets and at some point the two pairs will synchronise.

The problem is that if the frontend closes the socket without everything being synchronised, the latest changes are never persisted.

How could I pause the closing of the socket until the frontend document and the backend document are synchronised with the latest changes?

Temporary solution

When I detect that the socket is going to close, what I do is ask the backend if the document has the latest change that the frontend has.

To do this I use the getAllChanges function of automerge and see if the hash sent by the frontend exists.

But to do that I have to read all the changes in the document over and over again...

Solution

  • I am working in something similar ended up using the provided sync protocol. The plan is, accoding to the documentation, on every change to sent the message generated by generateSyncMessage.

    In case or resume you can replay all chages.

    Disclaimer: I have not succeeded on getting all chages in the syncMessage following the docs. Here is an example of how to generate the synceMessag based on docs:

     const dummyDoc = automerge.change(automerge.from(
            {content: ""}), doc => { doc.content = "dummy" })
 let dummyState = automerge.initSyncState()
 let msg: Uint8Array | null = null;
 [dummyState, msg] = automerge.generateSyncMessage(dummyDoc, dummyState)

    sending the msg over the network ... on the peer side:

    let msg: Uint8Array | null = null;
[peerDoc, peerSyncState, msg] = automerge.receiveSyncMessage(this._document, this._syncState, message);
if (msg) {
 // communicate that back to the first peer.
}

    In that way there should be some back and forth ... until the docs are complety synced...

    To make sure that changes are not lost on connection resets ... or net issues we can also add some buffering to the communication channels.

    Hope I helped.