Using the "using" statement with StreamWriter/Reader inside loop

Question

Here is my situation.

1. Read one line from a text file
2. "Process" the line
3. Write the "processed" line to a new text file
4. Loop to #1 and repeat until EOF

Here is how I'm doing it:

using (StreamReader srReader = new StreamReader(strInputFile))
{
    // loop until EOF
    while ((strCurrentLine = srReader.ReadLine()) != null)
    {
        // "process" strCurrentLine...

        // write the "processed" strCurrentLine to a new text file
        using (StreamWriter sw = new StreamWriter(strFullOutputPathFileName, false))
        {
            // write strCurrentLine to the new file
            sw.WriteLine("stuff...");
        }
     }
}

My manager tells me that using the using statement like I am inside of a loop will extremely hinder performance. The reason is because the StreamWriter instance will be created as many times as I'm looping. So, if I looped 1,000 times, I'd have 1,000 instances of my StreamWriter object, which could severely hinder performance.

Is this true? Also, is my method above the best way to accomplish this?

Answer 1

My manager tells me that using the using statement like I am inside of a loop will extremely hinder performance. The reason is because the StreamWriter instance will be created as many times as I'm looping. So, if I looped 1,000 times, I'd have 1,000 instances of my StreamWriter object, which could severely hinder performance.

Is this true?

Well, it's true, but not because you're creating instances, but because you're opening and closing a file 1,000 times. You could create 1,000 strings with almost no impact to performance.

Also, is my method above the best way to accomplish this?

To start, move the writer creation outside of the while loop:

using (StreamReader srReader = new StreamReader(strInputFile))
{
    // write the "processed" strCurrentLine to a new text file
    using (StreamWriter sw = new StreamWriter(strFullOutputPathFileName, false))
    {
        // loop until EOF
        while ((strCurrentLine = srReader.ReadLine()) != null)
        {
            // "process" strCurrentLine...

            // write strCurrentLine to the new file
            sw.WriteLine("stuff...");
        }
     }
}

However, you could also read the entire file into memory, process it, and write it out in one operation. The impact will depend on the processing that's done and whether you want partial results if there's an error.