Use MemoryMappedFile as BinaryReader

Question

I'm using MemoryMappedFile(MMF) to put large files in memory. Working with memory limit 32MB. 50MB file is loaded using MMF for 2-3 seconds. Reading data from MMF works fine and fast. The only problem for me is: I have big project with lot of using of BigEndianReader ( derived from BinaryReader) in multiple places. Instead of rewriting code - I prefer to modify this class to replace BinaryReader calls with MMF calls. Does anybody have idea how to create strteam from MMF? I have IntPtr for MMF but I dont understand how to create Stream from it.

Answer 1

You'll have to subclass Stream, keep track of the stream's Position and the currently mapped segment:

public class MmfStream : Stream
{
    private IntPtr currentMap;
    private long mapOffset;
    private long mapSize;
    private long position;

The Stream base class requires you to implement a Read and Write method, so whenever the application tries to read or write, you can use Marshal.Copy to copy data from the currently mapped segment.

If the stream's Position gets beyond the mapped segment, you create a new map and provide information from the newly mapped segment. You'll also have to deal with having to access data from the current and the newly mapped view. Something like:

public override int Read(byte[] buffer, int offset, int size)
{
    // current position not in current mapping?
    if (position < mapOffset)
    {
        // relocate the view
    }

    // compute how many bytes can be read from the current segment
    int read = Math.Min(size, (mapOffset + mapSize) - position);

    // copy those bytes to the buffer
    Marshal.Copy(...);

    position += read;

    if (size > read)
    {
        // recurse for the remaining bytes
        read += Read(buffer, offset + read, size - read);
    }

    return read;
}

(there are probably one-off errors in this code snippet; its just an illustration of the idea)