Search code examples
firebasegoogle-cloud-firestorebigdatamodeling

Saving to firestore objects more than 1MB

Hi I read that the limitation of the Firestore max document size is 1MiB.

I want to store heart rate and other "activity" data and my current model is similar to this

-activity
  - HR points stream (1 point per second up to 86400 points)
  - Lat points stream (same as above)
  - Long points stream (same as above)

The above number (86400), for example, is for 24h of storage.

In general, I am trying to eg store and get a let's say 24h run that someone did.

However, due to the firestore limitations on size, this is getting to predict.

Can you suggest or guide to the right direction of what would someone do in order to store such "big" data?

I have tried to separate the above streams to their own documents but with no luck as a stream of heartrate even can get too much data for the firestorm to store.

So in short what are the recommendations for storing big streams of data only with the firestore and not using for example cloud storage?

Solution

  • Can you suggest or guide to the right direction of what would someone do in order to store such "big" data?

    In order to store such "big" data you should change the way you are holding that data from within a single documents to a collection. In case of collections, there is no limitation. You can add as many documents as you want. According to the official documentation regarding Cloud Firestore Data model:

    Cloud Firestore is optimized for storing large collections of small documents.

    So you should take advantage of this feature.

    For details, I recommend you see my answer from this post where I have explained some practices regarding storing data in arrays (documents), maps or collections.