SSIS: OutOfMemoryException thrown while loading data from large table
I have a really large table that I am not able to extract. This table is extracted thanks to the proprietary database ODBC driver that I link into an ADO.NET Source.
The source table has 90,000 rows and 333 columns. I only convert 3 columns between extraction and loading into an Oracle database.
Everytime I run the package, it fails after 66,500 rows loaded with an OutOfMemoryException. I tried to change the batch size to really low value like 10 or 50 but it still fails.
Screenshot of my dataflow:
NB: For other tables from this database I had to lower batch size under 300 because the amount of column was exceeding the number of available parameters for generated insert queries and was causing the package to fail.
So I finally came up with a solution some times after posting this issue. I simply ended up setting the BatchSize to 1 on the destination for this particular package and it worked. This is the only package where I encountered the issue.
- Math.Sin() gives incorrect value
- How to run my python script when the sunOS is start booting
- Express-session: not resetting cookie expiration on each request
- Getting a stack overflow exception when normalizing a vector
- Edit default summary function in R gives error for multiple variables
- What was a For loop? Why isn't it needed in R?
- How to use download button in shiny and save results in various formats (csv, texte, pdf, spss...)?
- Why are there two assignment operators, `<-` and `->` in R?
- lm()$assign: what is it?
- How to get the value of list(...) in R and S functions
- Design matrix for MLM from library(lme4) with fixed and random effects
- how to generate elements not included in my sample
- Create a matrix with gradually changing values without a for loop
- Emacs ESS and S-plus ( S+ ) 8.1 compatability
- How to lag date-index in a time-series in R?
- Nonlinear regression in R / S
- Calling R from S-Plus?