I have an Ubunutu VM I set up with Google Compute, and because it was very nice I am trying to create a copy of this image and archive it, so I can create VM instances with it in the future. However, I am experiencing random command-line errors, so I will lay out all the steps I did to see if any are obviously wrong.
First of all, the VM, the image I created with it, image-1, and the Google storage bucket to try and store it can all be seen here.
I followed step 1 in creating an image, step 2 in creating a storage area that Google can store large objects like VM images for me, step 3 in installing Google Cloud SDK on my local computer to run the necessary commands, and followed step 4 in which commands to run.
After running
gcloud compute images export --destination-uri gs://bucket_address --image image-1
I get a long log that tells me an error happened.
Log:
Created [https://cloudbuild.googleapis.com/v1/projects/bayes-230019/builds/015e4129-5407-48cd-8fb5-0b0803406a0f].
Logs are available at [https://console.cloud.google.com/gcr/builds/015e4129-5407-48cd-8fb5-0b0803406a0f?project=297462970761].
[Daisy] Running workflow "image-export" (id=2d3br)
[image-export]: 2019-03-21T18:05:51Z Validating workflow
[image-export]: 2019-03-21T18:05:51Z Validating step "setup-disks"
[image-export]: 2019-03-21T18:05:51Z Validating step "image-export-export-disk"
[image-export.image-export-export-disk]: 2019-03-21T18:05:51Z Validating step "setup-disks"
[image-export.image-export-export-disk]: 2019-03-21T18:05:51Z Validating step "run-image-export-export-disk"
[image-export.image-export-export-disk]: 2019-03-21T18:05:52Z Validating step "wait-for-inst-image-export-export-disk"
[image-export.image-export-export-disk]: 2019-03-21T18:05:52Z Validating step "copy-image-object"
[image-export.image-export-export-disk]: 2019-03-21T18:05:52Z Validating step "delete-inst"
[image-export]: 2019-03-21T18:05:52Z Validation Complete
[image-export]: 2019-03-21T18:05:52Z Workflow Project: bayes-230019
[image-export]: 2019-03-21T18:05:52Z Workflow Zone: us-east4-c
[image-export]: 2019-03-21T18:05:52Z Workflow GCSPath: gs://bayes-230019-daisy-bkt/
[image-export]: 2019-03-21T18:05:52Z Daisy scratch path: https://console.cloud.google.com/storage/browser/bayes-230019-daisy-bkt/daisy-image-export-20190321-18:05:50-2d3br
[image-export]: 2019-03-21T18:05:52Z Uploading sources
[image-export]: 2019-03-21T18:05:52Z Running workflow
[image-export]: 2019-03-21T18:05:52Z Running step "setup-disks" (CreateDisks)
[image-export.setup-disks]: 2019-03-21T18:05:52Z CreateDisks: Creating disk "disk-image-export-image-export-2d3br".
[image-export]: 2019-03-21T18:06:37Z Step "setup-disks" (CreateDisks) successfully finished.
[image-export]: 2019-03-21T18:06:37Z Running step "image-export-export-disk" (IncludeWorkflow)
[image-export.image-export-export-disk]: 2019-03-21T18:06:37Z Running step "setup-disks" (CreateDisks)
[image-export.image-export-export-disk.setup-disks]: 2019-03-21T18:06:37Z CreateDisks: Creating disk "disk-image-export-export-disk-image-export-image-export--2d3br".
[image-export]: 2019-03-21T18:06:37Z Error running workflow: step "image-export-export-disk" run error: step "setup-disks" run error: googleapi: Error 403: Quota 'SSD_TOTAL_GB' exceeded. Limit: 100.0 in region us-east4., quotaExceeded
[image-export]: 2019-03-21T18:06:37Z Workflow "image-export" cleaning up (this may take up to 2 minutes).
[Daisy] Errors in one or more workflows:
image-export: step "image-export-export-disk" run error: step "setup-disks" run error: googleapi: Error 403: Quota 'SSD_TOTAL_GB' exceeded. Limit: 100.0 in region us-east4., quotaExceeded
ERROR
ERROR: build step 0 "gcr.io/compute-image-tools/daisy:release" failed: exit status 1
ERROR: (gcloud.compute.images.export) build 015e4129-5407-48cd-8fb5-0b0803406a0f completed with status "FAILURE"
It mentions disk size errors, but my image is only 31 Gb in size, and it says I am exceeding a 100Gb quota on my bucket located in region us-east4? What am I doing wrong?
Following the advice to upgrade my account, I did so, and now the code proceeds further along, but I get another weird error at the end.
image-export: step "image-export-export-disk" run error: step "copy-image-object" run error: error copying from gs://bayes-230019-daisy-bkt/daisy-image-export-20190321-20:14:58-mdxb2/outs/image-export-export-disk.tar.gz to gs://ml_vm_bucket: storage: object name is empty
New Log:
Created [https://cloudbuild.googleapis.com/v1/projects/bayes-230019/builds/02d7af72-12a4-4950-9bc6-153aabeef788].
Logs are available at [https://console.cloud.google.com/gcr/builds/02d7af72-12a4-4950-9bc6-153aabeef788?project=297462970761].
[Daisy] Running workflow "image-export" (id=mdxb2)
[image-export]: 2019-03-21T20:14:58Z Validating workflow
[image-export]: 2019-03-21T20:14:58Z Validating step "setup-disks"
[image-export]: 2019-03-21T20:14:58Z Validating step "image-export-export-disk"
[image-export.image-export-export-disk]: 2019-03-21T20:14:58Z Validating step "setup-disks"
[image-export.image-export-export-disk]: 2019-03-21T20:14:58Z Validating step "run-image-export-export-disk"
[image-export.image-export-export-disk]: 2019-03-21T20:14:59Z Validating step "wait-for-inst-image-export-export-disk"
[image-export.image-export-export-disk]: 2019-03-21T20:14:59Z Validating step "delete-inst"
[image-export.image-export-export-disk]: 2019-03-21T20:14:59Z Validating step "copy-image-object"
[image-export]: 2019-03-21T20:14:59Z Validation Complete
[image-export]: 2019-03-21T20:14:59Z Workflow Project: bayes-230019
[image-export]: 2019-03-21T20:14:59Z Workflow Zone: us-east4-c
[image-export]: 2019-03-21T20:14:59Z Workflow GCSPath: gs://bayes-230019-daisy-bkt/
[image-export]: 2019-03-21T20:14:59Z Daisy scratch path: https://console.cloud.google.com/storage/browser/bayes-230019-daisy-bkt/daisy-image-export-20190321-20:14:58-mdxb2
[image-export]: 2019-03-21T20:14:59Z Uploading sources
[image-export]: 2019-03-21T20:15:00Z Running workflow
[image-export]: 2019-03-21T20:15:00Z Running step "setup-disks" (CreateDisks)
[image-export.setup-disks]: 2019-03-21T20:15:00Z CreateDisks: Creating disk "disk-image-export-image-export-mdxb2".
[image-export]: 2019-03-21T20:15:43Z Step "setup-disks" (CreateDisks) successfully finished.
[image-export]: 2019-03-21T20:15:43Z Running step "image-export-export-disk" (IncludeWorkflow)
[image-export.image-export-export-disk]: 2019-03-21T20:15:43Z Running step "setup-disks" (CreateDisks)
[image-export.image-export-export-disk.setup-disks]: 2019-03-21T20:15:43Z CreateDisks: Creating disk "disk-image-export-export-disk-image-export-image-export--mdxb2".
[image-export.image-export-export-disk]: 2019-03-21T20:16:02Z Step "setup-disks" (CreateDisks) successfully finished.
[image-export.image-export-export-disk]: 2019-03-21T20:16:02Z Running step "run-image-export-export-disk" (CreateInstances)
[image-export.image-export-export-disk.run-image-export-export-disk]: 2019-03-21T20:16:02Z CreateInstances: Creating instance "inst-image-export-export-disk-image-export-image-export--mdxb2".
[image-export.image-export-export-disk]: 2019-03-21T20:16:11Z Step "run-image-export-export-disk" (CreateInstances) successfully finished.
[image-export.image-export-export-disk]: 2019-03-21T20:16:11Z Running step "wait-for-inst-image-export-export-disk" (WaitForInstancesSignal)
[image-export.image-export-export-disk.run-image-export-export-disk]: 2019-03-21T20:16:11Z CreateInstances: Streaming instance "inst-image-export-export-disk-image-export-image-export--mdxb2" serial port 1 output to https://storage.cloud.google.com/bayes-230019-daisy-bkt/daisy-image-export-20190321-20:14:58-mdxb2/logs/inst-image-export-export-disk-image-export-image-export--mdxb2-serial-port1.log
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:16:11Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": watching serial port 1, SuccessMatch: "ExportSuccess", FailureMatch: "ExportFailed:", StatusMatch: "GCEExport:".
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:16:22Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Running export tool."
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:16:22Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Disk /dev/sdb is 31 GiB, compressed size will most likely be much smaller."
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:16:22Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Exporting disk to gs://bayes-230019-daisy-bkt/daisy-image-export-20190321-20:14:58-mdxb2/outs/image-export-export-disk.tar.gz."
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:16:22Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Beginning copy..."
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:16:22Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Read 504 MiB of 31 GiB (101 MiB/sec), total uploaded size: 76 MiB (15 MiB/sec)"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:17:12Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Read 3.5 GiB of 31 GiB (68 MiB/sec), total uploaded size: 1.5 GiB (32 MiB/sec)"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:17:52Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Read 6.5 GiB of 31 GiB (69 MiB/sec), total uploaded size: 2.9 GiB (32 MiB/sec)"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:18:42Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Read 9.5 GiB of 31 GiB (68 MiB/sec), total uploaded size: 4.2 GiB (31 MiB/sec)"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:19:22Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Read 12 GiB of 31 GiB (54 MiB/sec), total uploaded size: 5.8 GiB (37 MiB/sec)"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:20:12Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Read 15 GiB of 31 GiB (65 MiB/sec), total uploaded size: 7.4 GiB (35 MiB/sec)"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:20:52Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Read 19 GiB of 31 GiB (102 MiB/sec), total uploaded size: 8.5 GiB (25 MiB/sec)"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:21:42Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Read 23 GiB of 31 GiB (93 MiB/sec), total uploaded size: 9.6 GiB (26 MiB/sec)"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:22:22Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Read 28 GiB of 31 GiB (95 MiB/sec), total uploaded size: 11 GiB (25 MiB/sec)"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:23:02Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": StatusMatch found: "GCEExport: Finished export in 6m43.907834062s"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2019-03-21T20:23:02Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--mdxb2": SuccessMatch found "ExportSuccess"
[image-export.image-export-export-disk]: 2019-03-21T20:23:02Z Step "wait-for-inst-image-export-export-disk" (WaitForInstancesSignal) successfully finished.
[image-export.image-export-export-disk]: 2019-03-21T20:23:02Z Running step "copy-image-object" (CopyGCSObjects)
[image-export.image-export-export-disk]: 2019-03-21T20:23:02Z Running step "delete-inst" (DeleteResources)
[image-export.image-export-export-disk.delete-inst]: 2019-03-21T20:23:02Z DeleteResources: Deleting instance "inst-image-export-export-disk".
[image-export]: 2019-03-21T20:23:02Z Error running workflow: step "image-export-export-disk" run error: step "copy-image-object" run error: error copying from gs://bayes-230019-daisy-bkt/daisy-image-export-20190321-20:14:58-mdxb2/outs/image-export-export-disk.tar.gz to gs://ml_vm_bucket: storage: object name is empty
[image-export]: 2019-03-21T20:23:02Z Workflow "image-export" cleaning up (this may take up to 2 minutes).
[Daisy] Errors in one or more workflows:
image-export: step "image-export-export-disk" run error: step "copy-image-object" run error: error copying from gs://bayes-230019-daisy-bkt/daisy-image-export-20190321-20:14:58-mdxb2/outs/image-export-export-disk.tar.gz to gs://ml_vm_bucket: storage: object name is empty
ERROR
ERROR: build step 0 "gcr.io/compute-image-tools/daisy:release" failed: exit status 1
ERROR: (gcloud.compute.images.export) build 02d7af72-12a4-4950-9bc6-153aabeef788 completed with status "FAILURE"
You are on the Free Tier. To increase your quota you will need to upgrade to a paid account.
Behind the scenes Google allocates SSD disk space for the workflow. This disk space is temporary so it does not affect your billing by very much, but free tier accounts cannot alloate enough space for the workflow.