Search code examples
google-bigquerygoogle-ads-api

BigQuery Data Transfer - Data Backfill vs Refresh


Can I know the difference between data backfilling and refreshing, in the context of BigQuery Data Transfers.

Data Refresh window is a parameter that can be configured when setting up BigQuery Data Transfers for Google Ads and Search Ads 360 data (probably for some other BigQuery data transfers too).

Reference:
https://cloud.google.com/bigquery-transfer/docs/working-with-transfers#refreshing https://cloud.google.com/bigquery-transfer/docs/adwords-transfer


Solution

  • -Data transfer:

    After you configure a data transfer, the BigQuery Data Transfer Service automatically loads data into BigQuery on a regular basis.

    I.e fetch data from last 24 hours to you BQ dataset.

    -Backfills:

    You can also initiate data backfills to recover from any outages or gaps.

    I.e fetch data gaps, insert new lines that wasn't imported in the data transfer.

    -Refresh: Overwrites a fresh copy of data in the destination table.

    source:

    Cloud Storage transfers  |  BigQuery Data Transfer Service  |  Google Cloud

    What is BigQuery Data Transfer Service?  |  Google Cloud