I am building a multi tenant app, where each tenant will have its own database with its own collections. All databases are stored in the same M10 cluster. For now, a tenant represents around 56 collections and 208 indexes. I have seen there is a recommended maximum for M10 cluster of 5000 collections and indexes (https://www.mongodb.com/docs/atlas/reference/atlas-limits/)
So if my understanding is correct, M10 cluster suits best for 18 maximum tenants (5000/(56+208)=18,93).
The documentation says The performance of a cluster might degrade if it serves a large number of collections and indexes. Does anyone have tried to exceed this limit? How big are these decreases in performance?
Apart from a hard limit on the number of collections and indexes you can have, the performance impact of having a large number of collections and indexes comes from the adverse impact of having too many data handles open. Not only this, but the maintenance also becomes a nightmare.
On the other hand, having large number of indexes will adversely impact the write operations in those collections, and will either continuously occupy the space in memory or if the memory is insufficient, will lead to continuous eviction and loading of indexes from and into the memory. To know more about the internal cache, see the official documentation here.
In conclusion, having more than 3500 indexes (for 18 tenants) and ~1000 collections will have a serious adverse impact on the performance of your overall cluster. You can monitor the same via the Cache Activity Metric, and others. Since it's anyway logical separation even if you create separate databases, you're advised to instead implement multi-tenancy via a tenant_id field in collections, instead of having different collections for different tenants.