Search code examples
javagoogle-app-enginegoogle-cloud-datastoresharding

Sharding counters with 180 properties


Does it sound bad to have 180 unindexed properties(columns) with Integer/Long type per entity in datastore?

I need to count 6 requests per user saving by day for analytics reasons and I'm doing everything based on the sharding counters article and webcast: https://cloud.google.com/appengine/articles/sharding_counters

So basically it's 6 values per day incrementing every new request, so I'm thinking in having:

1 Kind per Month 6 types of analytics * month days = 180

How much is too much in Google Datastore properties?

Thank you


Solution

  • Probably not a good idea.

    Keep in mind that every time you want to update a single property value the entire entity will have to be re-written (i.e. retrieved from the datastore, deserialized, updated, re-serialized and re-sent to the datastore). The bigger the entity, the slower the performance.

    IMHO it's better to have multiple smaller entities than a big one in such case. It is possible to split a single big entity into multiple smaller ones, efficiently related to each-other - see re-using an entity's ID for other entities of different kinds - sane idea?

    Along the same line I believe it's even possible to find a way to encode the day info and the user ID into unique custom key IDs, for easy access. Something like <userid>_YYMMDD or just <userid>_DD