Writing a text file into a gz file using GZipStream without first writing the text file to disk

Question

Im currently generating a large output from a few database queries. The resulting XML file is about 2GB. (This is a years worth of data). To save some disk space and download time for the client i am adding this file into a compressed file using the GZipStream class. See below for how i am currently compressing the file into a gz. NOTE: the fi object is a FileInfo.

using (FileStream inFile = fi.OpenRead())
using (FileStream outFile = File.Create(fi.FullName + ".gz"))
using (GZipStream Compress = new GZipStream(outFile, CompressionMode.Compress))
{
    byte[] buffer = new byte[65536];
    int numRead;
    while ((numRead = inFile.Read(buffer, 0, buffer.Length)) != 0)
    {
        Compress.Write(buffer, 0, numRead);
    }
}

This method works fine but requires me to write out the 2GB text file to disk and then read it all back in again in order to add it to the GZipStream and then write it back out again as a compressed file. It seems like a waste of time.

Is there a way to add my 2GB string directly to the GZipStream without first having to write to the disk?

Answer 1

If there's any way you can get the database result into a string and then load it into a MemoryStream you should be alright:

        var databaseResult = "<xml>Very Long Xml String</xml>";

        using (var stream = new MemoryStream())
        {
            using (var writer = new StreamWriter(stream))
            {
                writer.Write(databaseResult);
                writer.Flush();
                stream.Position = 0;

                using (var outFile = File.Create(@"c:\temp\output.xml.gz"))
                using (var Compress = new System.IO.Compression.GZipStream(outFile, CompressionMode.Compress))
                {
                    var buffer = new byte[65536];
                    int numRead;
                    while ((numRead = stream.Read(buffer, 0, buffer.Length)) != 0)
                    {
                        Compress.Write(buffer, 0, numRead);
                    }
                }
            }
        }