Search code examples
google-cloud-platformgoogle-cloud-datastoredatabase-migrationgoogle-cloud-spanner

How to migrate from Cloud Datastore to Cloud Spanner?


I'm looking at dataflow to try to export data from Datastore to Spanner. Spanner seems to only accept avro files via a dataflow connector. The only dataflow template I can find is from datastore to text-files. And the datastore export service is only exporting into levelDB format. Any clue would be helpful, Thanks!


Solution

  • The solution I ended up with was extracting the data as csv, then converting to Avro with PySpark. And finally inserting with Dataflow into Spanner.