Search code examples
sqlgoogle-bigquerysnapshotdbt

DBT snapshot command recreates table every time, does not add the dbt fields


I am working at my first snapshot prototype with dbt and BigQuery. It has the following configuration:

{{
        config(
          target_schema=REDACTED,
          strategy='check',
          unique_key=REDACTED,
          check_cols='all',
          materialized='table',
          cluster_by = REDACTED,
          labels = REDACTED
        )
    }}

In the beginning it worked fine, on the first dbt snapshot the table was created and it had the field dbt_valid_from and dbt_valid_to.

The first run showed:

[CREATE TABLE (228.0 rows, 21.4 KB processed) in 4.71s]

On the second run it showed:

[MERGE (0.0 rows, 37.7 KB processed) in 11.24s]

Then for some reason, this stopped working. Now every time I run dbt snapshot, the table is recreated from scratch. What's more, it doesn't have the dbt fields dbt_valid_from and dbt_valid_to. So it's understandable that dbt cannot make the snapshotting work.

I tried reversing all of my actions, but I can't get out of this problem. I also tried deleting the target folder with the compiled files. How can I get snapshotting to work again?


Solution

  • Apparently the problem was with adding materialized='table' in the snapshot configuration. I thought that without this, the snapshot would be a view. But that’s not the case. So I will omit this option altogether.