Search code examples
python-3.xgoogle-cloud-platformgoogle-cloud-functionsgoogle-cloud-spanner

How do I batch upsert data into Google Cloud Spanner using the Python client library?


I would like to upsert the contents of a pandas dataframe into a table in a Google Cloud Spanner database. The documentation here recommends using the insert_or_update() method of the batch object.

If the batch object is created by running this

from google.cloud import spanner_v1
client = spanner_v1.Client()
batch = client.batch()

Then this object does not have that method available. Running dir(client) gives me these results

['SCOPE', 
'_SET_PROJECT', 
'__class__', 
'__delattr__', 
'__dict__', 
'__dir__', 
'__doc__', 
'__eq__', 
'__format__', 
'__ge__', 
'__getattribute__', 
'__getstate__', 
'__gt__', 
'__hash__', 
'__init__', 
'__init_subclass__', 
'__le__', 
'__lt__', 
'__module__', 
'__ne__', 
'__new__', 
'__reduce__', 
'__reduce_ex__', 
'__repr__', 
'__setattr__', 
'__sizeof__', 
'__str__', 
'__subclasshook__', 
'__weakref__', 
'_credentials', 
'_database_admin_api', 
'_determine_default', 
'_http', 
'_http_internal', 
'_instance_admin_api', 
'_item_to_instance', 
'copy', 
'credentials', 
'database_admin_api', 
'from_service_account_json', 
'instance', 
'instance_admin_api', 
'list_instance_configs', 
'list_instances', 
'project', 
'project_name', 
'user_agent']

How do I do batch upsert in Spanner?


Solution

  • The snippets has an example of batch insert. I checked that the batch object created in the snippet also has an insert_or_update field.

    https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/spanner/cloud-client/snippets.py#L72

    ['class', 'delattr', 'dict', 'doc', 'enter', 'exit', 'format', 'getattribute', 'hash', 'init', 'module', 'new', 'reduce', 'reduce_ex', 'repr', 'setattr', 'sizeof', 'str', 'subclasshook', 'weakref', '_check_state', '_mutations', '_session', 'commit', 'committed', 'delete', 'insert', 'insert_or_update', 'replace', 'update']

    Can you try that out?