I am working at my first snapshot prototype with dbt and BigQuery. It has the following configuration:
{{
config(
target_schema=REDACTED,
strategy='check',
unique_key=REDACTED,
check_cols='all',
materialized='table',
cluster_by = REDACTED,
labels = REDACTED
)
}}
In the beginning it worked fine, on the first dbt snapshot
the table was created and it had the field dbt_valid_from
and dbt_valid_to
.
The first run showed:
[CREATE TABLE (228.0 rows, 21.4 KB processed) in 4.71s]
On the second run it showed:
[MERGE (0.0 rows, 37.7 KB processed) in 11.24s]
Then for some reason, this stopped working. Now every time I run dbt snapshot
, the table is recreated from scratch. What's more, it doesn't have the dbt fields dbt_valid_from
and dbt_valid_to
. So it's understandable that dbt cannot make the snapshotting work.
I tried reversing all of my actions, but I can't get out of this problem. I also tried deleting the target
folder with the compiled files. How can I get snapshotting to work again?
Apparently the problem was with adding materialized='table'
in the snapshot configuration. I thought that without this, the snapshot would be a view. But that’s not the case. So I will omit this option altogether.