Search code examples
amazon-web-servicesamazon-s3google-cloud-platformgoogle-cloud-storagegreat-firewall-of-china

Artifacts Migration from GCP non-China to AWS China region


I need to transfer my artifacts (zips and container images) stored in GCP us-west1 region on Cloud Storage and container registry to AWS China region S3 bucket and ECR.

Solution I found shows transfer of data from AWS non-China Account to AWS China account.

My question is :

  • Can I directly transfer artifacts using above solution (in link) from GCP to AWS (China) or do I have to transfer artifacts from GCP -> AWS (non-China) -> AWS (China)
  • Can the solution in link be implemented for any cloud or this is valid only for AWS ?

Solution

  • The gsutil tool on GCP could be used to perform this transfer, given that it supports transferring between cloud providers. According to the documentation, it should support any cloud provider storage service which uses HMAC authentication. After adding your credentials, you should be able to transfer files on Cloud Storage to AWS S3 using any available gsutil command combined with available wildcards. For example, this command should transfer every object from a GCP bucket to a S3 bucket:

    gsutil cp gs://GCS_BUCKET_NAME/** s3://S3_BUCKET_NAME
    

    While this should work for AWS buckets, I have not tested it with AWS buckets located in China. In case it does not work, you should first transfer the objects to a non-China AWS bucket, and then use the guide you have to move them to the China region.

    The process would not be much different when dealing with container images, as they are also stored in an automatically created Cloud Storage bucket. You need to review the permissions you have over this bucket, in case you run into permission errors. Otherwise, you can pull images from Container Registry into a local directory, and use the gsutil tool to transfer them into your S3 bucket:

    gsutil cp <source_dir> s3://S3_BUCKET_NAME