I am trying to deploy a MaskRCNN model to the GCP AI platform. The default online prediction doesn't work for me because my input size is bigger than the allowed limit. So, I decided to go ahead with the custom prediction route.
With respect to custom prediction, I am trying to use a larger machine-type than what is provided by default because my model is big and needs more memory than what the default one ( mls1-c1-m2 ) provides.
I want to deploy the model with n1-standard-4 machine type, and following is the command I am running:
gcloud beta ai-platform versions create $VERSION_NAME --model $MODEL_NAME --runtime-version 1.14 --python-version 3.5 --origin $MODEL_DIR --package-uris gs://pred_02/staging/my_custom_code-0.1.tar.gz --framework tensorflow --prediction-class predictor.MyPredictor --machine-type n1-standard-4
but this throws an error:
ERROR: (gcloud.beta.ai-platform.versions.create) FAILED_PRECONDITION: Machine type n1-standard-4 only support FRAMEWORK_TENSORFLOW
I tried different variations of --framework arg as well but all of them throw some similar error. So, why is it throwing this error?
If this works, then my idea is to use this machine type with a k80 accelerator. And this would be my second question? can I run it with an accelerator for custom prediction?
--prediction-class does not work on n1 machine types yet. You could use mls1-c4-m2 machines till then. Check for machine types here to learn about the constraints for each machine type.
Custom prediction routines are still evolving(beta). You should check for updates here.