how to handle download stream

Question

I need to download xml file from a secure link, and store the content into database.

Can i use text reader??? or i need to store the file into my local file system first, then read the content from my file system and store into my database?

HttpWebRequest downloadRequest = (HttpWebRequest)WebRequest.Create("https://line-to-xml-file.xml");
string Content;

                downloadRequest.Credentials = new NetworkCredential()
                                              {
                                                  UserName = this._userCredentials.UserName,
                                                  Password = this._userCredentials.Password
                                              };

                downloadRequest.PreAuthenticate = true;

                using (HttpWebResponse downloadHTTPResponse = (HttpWebResponse)downloadRequest.GetResponse())
                {
                    using (Stream downloadResponseStream = downloadHTTPResponse.GetResponseStream())
                    using (TextReader tReader = new StreamReader(downloadResponseStream))
                    {
                        Content = tReader.ReadToEnd();
                    }
                }
              return Content;

Since the remote file is huge, up to 100MB, i can see nothing from debug. and when i try to save it.

using (TransactionScope trans = new TransactionScope()) <--- when comes to this line, exception throws...
{
// perform update, save the content into databse
// send a notification message to message bus, indicate content has been updated
}

It complain MSDTC transaction timeout/cancelled

Answer 1

By having it into a stream you should be fine... if you have any problem with that specific stream then you could use a MemoryStream instead of a FileStream and use it the same way. But I hardly doubt this is your case.

I think you should make also sure to open the connection RIGHT BEFORE you are going to save the stream and when you have it completely loaded... You can also play with your Command.TimeOut property if it is taking really, really long to save, here "A value of 0 indicates no limit" but this should be avoided. but