Search code examples
sql-serveretldata-integration

Why do a periodic full data refresh?


Is there a benefit from doing a periodic full table refresh when you regularly insert/update/delete incrementally?

To clarify, this question is in regards to ETL processes.


Solution

  • If you are 100% certain that your incremental updates are capturing all CRUD operations, there is no reason to flush and fill. If your incrementals have room for error beyond the tolerance of the business rules governing the process, then you should consider period flush and fills.

    It all depends on your source system, your target system, your ETL process, and your tolerance for error.