Search code examples
c#filestreamlarge-files

Difficulty reading large file into byte array


I have a very large BMP file that I have to read in all at once because I need to reverse the bytes when writing it to a temp file. This BMP is 1.28GB, and I'm getting the "Out of memory" error. I can't read it completely (using ReadAllBytes) or using a buffer into a binary array because I can't initialize an array of that size. I also can't read it into a List (which I could then Reverse()) using a buffer because halfway through it runs out of memory.

So basically the question is, how do I read a very large file backwards (ie, starting at LastByte and ending at FirstByte) and then write that to disk?

Bonus: when writing the reversed file to disk, do not write the last 54 bytes.


Solution

  • With a StreamReader object, you can Seek (place the "cursor") to any particular byte, so you can use that to go over the entire file's contents in reverse.

    Example:

    const int bufferSize = 1024;
    string fileName = 'yourfile.txt';
    
    StreamReader myStream = new StreamReader(fileName);
    myStream.BaseStream.Seek(bufferSize, SeekOrigin.End);
    
    char[] bytes = new char[bufferSize];
    while(myStream.BaseStream.Position > 0)
    {
        bytes.Initialize();
        myStream.BaseStream.Seek(bufferSize, SeekOrigin.Current);
        int bytesRead = myStream.Read(bytes, 0, bufferSize);
    }