Search code examples
c#streamfilestream

Copy files with a chunked stream causes the files to be different sizes due to last read


Could someone be kind enough to explain how I get my files the same size after copying it using a chunked stream? I presume it is because the last chunk still has a buffer size of 2048 so it is putting empty bytes at the end, but I am unsure how I would adjust the last read?

Original size: 15.1 MB (15,835,745 bytes) New size: 15.1 MB (15,837,184 bytes)

        static FileStream incomingFile;

        static void Main(string[] args)
        {
            incomingFile = new FileStream(
             @"D:\temp\" + Guid.NewGuid().ToString() + ".png",
               FileMode.Create, 
               FileAccess.Write);

            FileCopy();
        }

        private static void FileCopy()
        {

            using (Stream source = File.OpenRead(@"D:\temp\test.png"))
            {

                byte[] buffer = new byte[2048];

                var chunkCount = source.Length;

                for (int i = 0; i < (chunkCount / 2048) + 1; i++)
                {
                    source.Position = i * 2048;
                    source.Read(buffer, 0, 2048);
                    WriteFile(buffer);
                }
                incomingFile.Close();
            }

        }
        private static void WriteFile(byte[] buffer)
        {

            incomingFile.Write(buffer, 0, buffer.Length);
        }

Solution

  • The last buffer read does not necessary contain exactly 2048 bytes (it can well be incomplete). Imagine, we have a file of 5000 bytes; in this case will read 3 chunks: 2 complete and 1 incomplete

    2048 
    2048  
     904 the last incomplete buffer
    

    Code:

            using (Stream source = File.OpenRead(@"D:\temp\test.png"))
            {
    
                byte[] buffer = new byte[2048];
    
                var chunkCount = source.Length;
    
                for (int i = 0; i < (chunkCount / 2048) + 1; i++)
                {
                    source.Position = i * 2048;
    
                    // size - number of actually bytes read 
                    int size = source.Read(buffer, 0, 2048);
    
                    // if we bytes to write, do it
                    if (size > 0)
                        WriteFile(buffer, size);
                }
                incomingFile.Close();
            }
    
            ... 
    
            private static void WriteFile(byte[] buffer, int size = -1) 
            {
               incomingFile.Write(buffer, 0, size < 0 ? buffer.Length : size);
            }
    

    In your case you write 15837184 == 7733 * 2048 bytes (7733 complete chunks) when you should write 15835745 == 7732 * 2048 + 609 bytes - 7732 complete chunks and the last incomplete one of 609 bytes