Search code examples
google-cloud-platformgoogle-bigquerygoogle-ai-platform

How to flush data while inserting in BigQuery?


I am inserting a row into BigQuery using the following code:

rows = [self.__dict__]
errors = client.insert_rows_json(table_id, rows)

It is being inserted into the streaming buffer and I am able to fetch from it. I want to modify a field of this data. How do I flush the data into the DB?


Solution

  • Posting @Roar S, Suggestion for better visibility for the community. @Roar S. is correct BQ supports update query but not recommended for large scale transactions. BigQuery is an OLAP database where it is focused on analytical and reporting purposes while the latter OLTP is more used for transactional processing.

    This I believe a good reference for comparison of both databases: https://www.googlecloudcommunity.com/gc/Technical-Tips-Tricks/DATABASE-Concept-Database-structures-OLTP-vs-OLAP/ta-p/592445/jump-to/first-unread-message