Search code examples
google-app-enginecopyadmindelaydatastore

Using the appengine datastore admin to move data to a new application - how long should this take?


I am moving to the high-replication datastore which is relatively simple when using the AppEngine Datastore Admin feature to copy all data from the old application to the high-replication application.

However, I have several applications (and started by copying applications with smaller databases), and have seen that the rate of copy is about 1Gb every two hours. This is acceptable for some of my applications, however I am now going to be moving an application with about 14GB of data, and would prefer to not have to shut down my website for more than a day while transferring the data.

Is there any way to make this process go faster? Are these data transfer speeds typical for the Datastore Admin tool?

Kind Regards


Solution

  • Google are working on a new migration tool, which will allow a shorter read-only period while migrating.

    They mentioned it when 1.5.3 was released:

    http://googleappengine.blogspot.com/2011/08/app-engine-153-sdk-released.html

    You can try signing up for early access to the migration tool here:

    https://docs.google.com/spreadsheet/viewform?authkey=CLXR0LMN&formkey=dERMcDZuMnlycHoyZDd4Vy1PNXlhWlE6MQ&ndplr=1