I am going to import a 30 GB delimited text file into SQL Server using Import and Export Wizard. the delimited character is | (pipeline) I've search every where on the Internet but no solution yet. there are some problems same as me but not the exact error message:
- Executing (Error)
Messages
Error 0xc020209c: Data Flow Task 1: The column data for column "Username" overflowed the disk I/O buffer.
(SQL Server Import and Export Wizard)
Error 0xc0202091: Data Flow Task 1: An error occurred while skipping data rows.
(SQL Server Import and Export Wizard)
Error 0xc0047038: Data Flow Task 1: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.
The PrimeOutput method on component "Source - Sessions_txt" (1) returned error code 0xC0202091.
The component returned a failure code when the pipeline engine called PrimeOutput().
The meaning of the failure code is defined by the component,
but the error is fatal and the pipeline stopped executing.
There may be error messages posted before this with more information about the failure.
(SQL Server Import and Export Wizard)
I tried to first import it to an excel file, then import the excel file into SQL Server, but the rows count is about 300,000,000 records!
I believe the problem is in initial import dialog setting in SQL Server Import and Export Wizard. I appreciate any solution you suggest.
Thanks.
I personally would probably write a Console application to read the file line by line and perform a bulk copy every X thousand lines. The program could record its progress so that if an error happens half way through the file you could fix the issue and continue processing again without the need to start from scratch. It's a pretty simple program to write so you could knock something together pretty fast.