I'm doing some planning for crisis situations (last time we went from 4k visitors a day to 1.3M) and I notice that the lower end of the Azure AppFabric cache has some pretty low simultaneous connection limits
128Mb & 256Mb cache =10 concurrent connections for instance.
I'm using cache for webrole session state - but only putting things in it in a very limited set of circumstances (live site has peaked at 0.03MB last month!) How do I figure out the maximum number of connections - is that going to be equivalent to the number of servers I have pointing at it ? The number of CPU's?
I've not tried it yet, but scaling the cache looks like it might be a 24hr operation???, so not responsive enough for scaling up on emergency demand.
Just after some guidelines to help me pick and initial cache size and to scale sensibly.
The number of connections you have to the cache is basically the number of DataCacheFactory instances you have. For this reason it is good practice to have as few instances of these that you can. You also need to be sure that when any Azure instance that has initialised a DataCacheFactory should dispose of the instance when it stops, this helps clear up the connections that it has open to the service.
However you said that you are using the cache as a session provider. This creates its own DataCacheFactory for each instance of the role. So basically one connection per role. Having said that the session provider appears to be a little lax with how it cleans up connections it doesn't use, so it's best to over provision the number of connections.
The other thing that you need to watch is the "Transactions Per Hour" limit on the cache. If every request to a page needs to access session information, this is how many page requests you can serve in an hour.
Resizing you cache is actually pretty quick, it generally only takes a minute or so. But you can only change once in every 24 hour period. So if you have increased load you can make the cache larger, but if the load is too much for the even the increased cache, then you need to wait for the 24 hours to be up to change it again. So you're probably better off making the cache much larger the first time you resize it and shrink it back down the next day.
EDIT:
While this information was correct at the time of writting, the November 2011 (1.6) update to the SDK has introduced cache connection pooling which is turned on by default if you're not configuring the cache through code. This makes it less crucial to have just one static DataCacheFactory and means that if you want to use the same connection information for both session and application data, this could all be one connection. More details can be found on MSDN.