I am faced with a challenge I would love to get some pointers for.
I have a couple of very large tables housed in an SQL Server instance that is also the primary production table for the application it supports.
I want to load 2 years worth of historical data, after which I will be implementing incremental refresh. The problem is that when I tried to do the initial load, the end users of the application experienced time outs and all the other stuff we want to see in production.
I am looking for a way to feed the data flow either in small steps of a month or smaller, or provide the historical data separately via a .csv or in some other way.
Can anybody share some insight on how to go about this? I've tried researching this issue, but I've not found a way so far.
Thank you in advance!
If you dont have the option to mirror your data i would suggest a dirty read.
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
BEGIN TRANSACTION
-- your sql script
COMMIT TRANSACTION