How can this data model be designed that does not hit the 20gb logical partition limit in Cosmos DB?

Question

I am going with a single table design for a chat application in Azure Cosmos db. I have my access patterns setup, just want to check here if I am in the right direction.

Below are the few things that concerns me

1. I have heard about a hard limit of 20gb per logical partition, wondering my below data model would eventually hit that limit.

2. Also is the below model considered hot partitioning as I am assuming a lot of focus will be there in conversationid-1234 partition as the messages in the conversation grows?

Access patterns -

1. Get details of the user -

   SELECT * from c WHERE c.pk = 'userid-1234' AND type = 'user'.

2. Get conversations the user is part of -

   SELECT * from c WHERE c.pk = 'userid-1234' AND type = 'member-of'

3. Get details of a conversation -

   SELECT * from c WHERE c.pk = 'conversationid-1234' AND type = 'conversation'

4. Get members of a conversation -

   SELECT * from c WHERE c.pk = 'conversationid-1234' AND type = 'member'

5. Get messages of a conversation -

   SELECT * from c WHERE c.pk = 'conversationid-1234' AND type = 'message'

6. Get pinned messages of a conversation -

   SELECT * from c WHERE c.pk = 'conversationid-1234' AND type = 'pinned-message'

7. Get reactions to a message in a conversation -

   SELECT * from c WHERE c.pk = 'conversationid-1234' AND type = 'message-reaction'

8. Get status of a message(read/unread) in a conversation -

   SELECT * from c WHERE c.pk = 'conversationid-1234' AND type = 'message-status'

Please advise.

Answer 1

Design questions like these are exceedingly difficult to answer. Usually because there is not enough detail in the question and that's mostly the case here. It's impossible to answer whether your design will hit the 20GB limit.

Access patterns to design partition strategies should include all operations, not just queries. They should also include the volume of operations and relative percentage to the others to understand which are most important.

You didn't explain why your design centers around a single container. I'm not sure if this is because you've tested this design under load and it makes sense, or if there is some other reason. Any data model and partitioning strategy should be tested under load against the top access patterns identified for the workload.

To understand whether your design will work with 20GB partitions, I recommend spending more time thinking through what operations will happen and in what numbers. You should also understand min/max/avg for payload sizes to allow you to estimate logical partition sizes given a number of users, number of chat messages, etc. over time. Apply pareto principle. You're trying to solve for the 80%, not every case or edge cases.

If you're looking for a simple chat app design to start from you can start with one I recently published here https://github.com/AzureCosmosDB/cosmos-chat