Search code examples
google-cloud-datalab

Deploying Google Cloud Datalab fails with "Not enough VMs ready"


I changed to a new project after I had an issue with a non-US project but now I have a different error problem.

I'm now in a us-east project zone, billing enabled.

The issue this time is detailed in the logs here:

Oct 18 10:43:59 datalab-deploy-main-20151018-10-34-58 startupscript: 10:43 AM Not enough VMs ready (0/1 ready, 1 still deploying). Deployed Version: datalab:main.387933421330143384
Oct 18 10:43:59 datalab-deploy-main-20151018-10-34-58 startupscript: ERROR: Not enough VMs ready (0/1 ready, 1 still deploying). Deployed Version: datalab:main.387933421330143384
Oct 18 10:43:59 datalab-deploy-main-20151018-10-34-58 startupscript: 10:43 AM Rolling back the update.  This can sometimes take a while since a VM version is being rolled back.
Oct 18 10:44:00 datalab-deploy-main-20151018-10-34-58 startupscript: Could not start serving the given version.
Oct 18 10:44:00 datalab-deploy-main-20151018-10-34-58 startupscript: ERROR: (gcloud.preview.app.deploy) Command failed with error code [1]
Oct 18 10:44:00 datalab-deploy-main-20151018-10-34-58 startupscript: Step deploy datalab module failed.

I tried again after a few weeks, still got the same error. Here are the logs:

    Nov 10 10:01:10 datalab-deploy-main-20151110-09-59-21 startupscript: If this is your first deployment, this may take a while...#015If this is your first deployment, this may take a while.../#015If this is your first deployment, this may take a while...-#015If this is your first deployment, this may take a while...\#015If this is your first deployment, this may take a while...|#015If this is your first deployment, this may take a while...done.
    Nov 10 10:01:10 datalab-deploy-main-20151110-09-59-21 startupscript: 
    Nov 10 10:01:10 datalab-deploy-main-20151110-09-59-21 startupscript: Building and pushing image for module [datalab]
    Nov 10 10:01:49 datalab-deploy-main-20151110-09-59-21 startupscript: Updating module [datalab]...#015
Updating module [datalab].../#015
Updating module [datalab]...-#015
Updating module [datalab]...\#015
Updating module [datalab]...|#015
Updating module [datalab].../#015
Updating module [datalab]...-#015

repeated lots and lots until finally...

startupscript: ERROR: (gcloud.preview.app.deploy) Not enough VMs ready (0/1 ready, 1 still deploying). Version: datalab:main.388466283677388941
Nov 10 10:08:22 datalab-deploy-main-20151110-09-59-21 startupscript: Step deploy datalab module failed.

Trying again in a totally fresh new project I now get a permissions error:

Nov 10 21:48:41 datalab-deploy-main-20151110-21-42-57 startupscript: Verifying that Managed VMs are enabled and ready.
Nov 10 21:48:41 datalab-deploy-main-20151110-21-42-57 startupscript: If this is your first deployment, this may take a while...#015If this is your first deployment, this may take a while...done.
Nov 10 21:48:41 datalab-deploy-main-20151110-21-42-57 startupscript: WARNING: If this is your first deployment, please try again.
Nov 10 21:48:41 datalab-deploy-main-20151110-21-42-57 startupscript: ERROR: (gcloud.preview.app.deploy) Server responded with code [403]:
Nov 10 21:48:41 datalab-deploy-main-20151110-21-42-57 startupscript:   Forbidden Unexpected HTTP status 403.
Nov 10 21:48:41 datalab-deploy-main-20151110-21-42-57 startupscript:   You do not have permission to modify this app (app_id=u's~random-bananas-111111').
Nov 10 21:48:42 datalab-deploy-main-20151110-21-42-57 startupscript: Step deploy datalab module failed.

Solution

  • None of the suggestions here helped, but after the 4th new project and attempting to deploy again, persistence proved to be the answer and it worked :)