With Google releasing Vertex AI lately that integrates all its MLOps platforms, I wonder what would be the difference in serving a custom trained PyTorch/Tensorflow model on GKE vs Vertex AI (or AI Platform Unified, since the rebranding just took place and AI Platform already provides the capability to serve model prediction).
I did a lot of research but found little info on this. I'm already hosting my ML model on GKE and is it worth it to migrate to Vertex AI?
Note: I'm not planning to do training and other data preprocessing on cloud yet.
It is worth considering Vertex AI as:
Vertex AI is a “Managed” ML platform for practitioners to accelerate experiments and deploy AI models. We don't need to manage the infrastructure/ servers/ health while deploying/training/ predicting ML models. Vertex AI will take care of that for you along with scaling according to the traffic.
Some key features which helps in considering Vertex AI:
Once the model is trained we get detailed model evaluation metrics and feature attributions. (Feature attribution tells which features in the model signaled model’s predictions the most, which gives insights into how model is performing under the hood)
Once the model is trained, it can be deployed to an endpoint. Traffic between models can be split for testing and machine type can also be customised
Vertex Pipelines help to avoid Concept of model drift which can happen when the environment around your model changes. Vertex pipeline can help automate this retaining workflow.
Vertex model monitoring can be used to detect things like drift and training-serving-skew, so rather than manually checking to make sure the model is still performing correctly, using vertex AI provides confidence in model reliability because we'll be alerted anytime something changes.