Recommended use of records in deepstream.io

Question

I have been using records recently and I was unsure of the practical limitations in terms of the overall size of the json structure. Is there any recommended max-length, e.g. could you store a whole chat history as an (anonymous) record, with maybe hundreds of single entries?

Probably the json document structure would look like this:

var record = client.record.getRecord( 'chat/5ak1g' );

record.set({
    2016.03.25.16:22:25: {
        user: 'Ann',
        message: 'Hey, whats up?'
    }
    2016.03.25.16:22:40: {
        user: 'Sue',
        message: 'Wanna get some sushi?'
    }
    2016.03.25.16:23:10: {
        user: 'Ann',
        message: 'cdn.example.com/sj48s2f4.jpg'
    }
  //more messages
});

So here's the question: Is a record in terms of size the appropriate solution for a problem like this or is it more fitting to use lists/RPC?

Thanks in advance!

Answer 1

There is a hard limit of 4MB per message, but this should be sufficient for even the longest chat history. The problem is more that a record is an atomic unit in deepstream - meaning you can't load half a record (they do however send deltas for updates). When it comes to storing (possibly very long) chat histories I think there are two alternatives:

A) If your messages remain mutable (e.g. a user can edit a message after it has been sent) create a record per message and store the record names in a list. There is an open issue about adding pagination to make handling large lists more efficient.

B) If your chat history is immutable but you want to keep a large amount of histories for a long time, you might build it as follows:

• Use events for chat messages
• Build a backend process that listens for events from any chat and stores them in a database (e.g. ds.event.listen( 'chat-message/(.*)', () => {} );)
• Add an RPC to retrieve specific parts of your chat history