Search code examples
cntk

CNTK deployment for real time predictions

TensorFlow has a separate project for its production usage, as noted here, called TensorFlow Serving.

How should I use CNTK in a production environment, and how should I handle it's deployment? Hopefully one could deploy trained models in to a server/cluster that serves the model with RPC or HTTP REST API.

If no such tool exists, what should be the first steps to develop it and a good architecture to roll out on my own?

Solution

  • We do support serving CNTK models in a production environment. You can find information about model evaluation/inference: https://github.com/Microsoft/CNTK/wiki/CNTK-Evaluation-Overview. For deployment, you need deploy the dlls specified here. A tutorial for deploying CNTK in Azure is available here.