I have a fact table for which I am getting data from a source A
.
Then, I have source B
which has one common field with the source A
. So, I wonder how I should populate the fact table with the keys from the dim tables?
It is not just a simple data pull as it requires some serious cleaning. I assume that in the staging tables I should not do any transformation. Does it mean that it would look like the following?
B
to the staging tables.Every day truncate the table in (2) and re-create the new table - see (2)?
You had a right direction, but I am usually separate all levels(staging, etl,dwh) because it's easier to control the data, clean and verify.
So here's what I would do: