When publishing an Azure ML Web Service and preloading data in our R model we see inconsistent performance. First calls are slow but following calls are fast, waiting a bit (couple of minutes) for the next call ends up showing longer response times.
The way Azure ML Web Services work in the background means that instances hosting the models are provisioned and moved in a very dynamic multi-tenant environment. Caching data (warming up) can be helpful but this doesn't mean all subsequent calls will land on the same instance with the same data available in the cache.
For models that need a lot of in-memory data there is a limit to what the Azure ML Web Services hosting layer can offer at this point. Microsoft R server could be an alternative to host these big ML workloads and looking at Service Fabric to scale