I am having difficulties figuring out if APPFabric caching or SQL Server should be used in the context of our needs (considering the fact that we are currently using SQL server for most things).
We only need (for now) to cache small chunks of data (~16KB) each of them corresponding to the information associated to a particular request that was sent from one of the applicative server (outgoing request).
Any of the applicative server can receive incoming request associated to the initial outgoing request, and for our needs we need to find back the original information associated with this outgoing request ... that's why we can't keep a local in memory cache in each applicative server, because we can't be sure the incoming request will arrive on the applicative server from which the outgoing request was sent.
HOWEVER we just basically need to persist the ~16kb piece of information only once (very rare possibilities of updates), and to be able to access it back from any applicative server, but to access it only one time in the vast majority of cases. So basically most of the time it will be one write from an applicative server (caching) and later on, one read from same or another applicative server.
In this specific context, will there be any gain of going through an AppFabric cache cluster instead of directly going to the database (considering it will be a simple insert/select statement) ?
Keeping in mind scalability, meaning that we currently do not have a high throughput of put_data / get_data operations (~160ops/sec) but we may reach 1K/s .. 10k/s and maybe more in the near future.
Thanks in advance for your answers.
The gain of AppFabric Cache versus SQL DB would be access time. You will have quicker access time for AppFabric since it stores everything in memory (RAM) whereas SQL needs to query its data from disk.
The downside of the AppFabric Cache is that you can lose the data unless you've implemented HA (high-availability) in your cluster to avoid data loss when systems fails. The SQL DB wins here because it supports data recoverability (via backup logs - LDFs) should the database system fail.
If you need guaranteed message delivery, you probably shouldn't use the AppFabric Cache cluster, but a SQL DB for temporary persistence due to its robust data recovery support.