Write large data into MemoryStream with C#

Question

I have a big stream (4Go), I need to replace some character (I need to replace one specific character with 2 or 3 ones) in that stream, i get the stream from à service à.d I have to return back a stream. This is what I'm doing

private static Stream UpdateStream(Stream stream, string oldCharacters, string newCharacters, int size = 2048)
{
    stream.Position = 0;

    StreamReader reader = new StreamReader(stream);

    MemoryStream outputStream = new MemoryStream();
    StreamWriter writer = new StreamWriter(outputStream);
    writer.AutoFlush = true;

    char[] buffer = new char[size];

    while (!reader.EndOfStream)
    {
        reader.Read(buffer, 0, buffer.Length);
        if (buffer != null)
        {
            string line = new string(buffer);
            if (!string.IsNullOrEmpty(line))
            {
                  string newLine = line.Replace(oldCharacters, newCharacters);
                writer.Write(newLine);
            }
        }
    }

    return outputStream;
}

But I'm getting an OutOfMemory exception at some point in this line but when looking at computer memery I still have planty available.

writer.Write(newLine);

Any advise ?

Answer 1

So to make it work I had to :

• use the HugeMemoryStream class from this post Failed to write large amount of data to stream
• and define the gcAllowVeryLargeObjects parameter to true

• and set the build to 64 bit (Prefer 32-bit unchecked)