Search code examples
google-cloud-platformgoogle-cloud-firestoregoogle-cloud-datastore

Google Datastore/Firestore - generational timestamp


Is it scalable to index a daily timestamp on an entity that only increases in value once per day? In essence, similar to a generation field as explained in the best practice docs here (although that doc link focuses on deletes, I am focused on writes). The field would have a value of 1 on day one, 2 the next day, and so on. Then you could sort on that generation field to get entities sorted by day.

Are there any hotspot issues for such a model? Because technically it is a monotonically increasing field, but the value only increases once per day.

EDITS:

Consider an entity that looks like this:

Task
 tag: String
 generation: Integer

Elsewhere in the application is a static global Integer set to a value of 1.

static int GLOBAL_VARIABLE = 1;

The value of this global counter increases by 1 every day at a specific time, perhaps by deploying a new application with the hardcoded value updated.

When writing a Task entity, the generation field gets set to that global integer.

Now you can query for all Task entities with a given tag and generation. You could also sort by generation, giving you Tasks that are from today at the top of the list and subsequent days towards the bottom of the list.

Would something like this be scalable? Or does this run into problems? Let's assume you are writing very frequently, more than 500 times per second.


Solution

  • Answering this as a community wiki. As Doug Stevenson mentioned in comments

    you have a generation field not a timestamp where performance is an issue In your scenario you have the generation field increasing once per day and is written more than 500 times per second, all the records for the given day have the same generation value. So Writing the same value during the burst would not cause an index partition range to have to grow at all.