I'm designing an architecture for leveraging Azure Search for multiple tenants. Since each tenant will have a slightly different schema my solution will require 1 index per tenant. This is easy enough to set up and I'm really liking what Microsoft has put together. However now that I am starting to think about on-boarding new tenants, monthly costs and scaling up the service I am starting to hit a few walls and wondering what my "best" option is.
Has anyone encountered this situation that can shed some light onto best practices? Here are the options as I see them now:
Option 1: Spin up a new BASIC plan for every 5 tenants at a cost of $38/m for every 5 tenants ($7.60 per tenant per month).
Pros: Cheap to start. Cons: Tenants are crippled by the limited performance and storage capabilities, I'll have to manage X number of services and ClientQueryKeys once I get past 5 indexes/tenants.
Option 2: Spin up a new STANDARD S1 plan for every 50 tenants at a cost of $250/m for every 50 tenants ($5 per tenant per month).
Pros: Better performance, less services to manage as tenant counts increase Cons: Much higher costs to start, still need to manage tenant-to-service relation once system has greater than 50 tenants, I'll have to manage X number of services and ClientQueryKeys once I get past 50 indexes/tenants.
Option 3: Spin up a single STANDARD S2 plan that can be used for ALL tenants (assuming no cap on index count)
Pros: Better performance, no need to manage multiple services/client keys as tenant counts increase Cons: Much higher costs to start, very little documentation on costs and limitations.
In all scenarios (aside from option 3, I'm assuming?) I would have to manage client keys across multiple services. So obviously having only one service with an infinite index count is ideal. However I am a startup (yes I am using BizSpark already) and the costs for search a very daunting when I may only have 1-5 tenants to start.
I've read that there is no way to easily migrate data between plans (without doing it manually or writing a script) so my first choice is likely to be my last. I would also prefer to only have to manage one service with one plan for all my tenant. Therefore I am leaning to option 3.
If option 3 is the best option:
S2 services work much better in multi-tenant scenarios. Not only they can fit more indexes (up to 200), but they also have more overall capacity so assuming exponential distribution of index sizes and loads, you get a better typical experience for your customers.
You're right that the cost of entry is higher.
Regarding the cons of S2, soon we're going to publish proper documentation and other supporting materials for it. In the meanwhile, feel free to contact me directly (Pablo DOT Castro AT the usual Microsoft domain) for more details.
If you think you'll have lots of indexes in the future (many 100s), we're also working on options for better multi-tenant support. We're not ready to announce the details yet but I'm happy to discuss if you get in touch with us.
Answering your specific questions:
1.Can I start on BASIC and scale up to S1 then S2 as needed, or is this not possible?
We don't currently support this. You'd have to create a new search service and migrate the indexes.
2.If BASIC cannot scale to S1 is it at least possible to scale from STANDARD S1 to S2 once I go past 50 tenants or will I need to manually manage this or start at S2?
No, it's not. We want to do this, just have not gotten to it yet.
3.What are my startup costs and/or costs per index/tenant on Standard S2?
Please get in touch with us and we can discuss pricing.
4.Is my index limit infinite on S2? 5.If not, what is the index cap?
No, S2 services are limited to 200 indexes/service.
6.Are there any other options or caveats that I should consider?
You've done a good analysis, I think you're on the right track. One thing you may want to consider is fairness. All indexes in the same service share the capacity you've provisioned for the service. If there's risk of unfair loads you might want to consider per-tenant throttling.