I'm really trying to understand how to use memory-mapped files properly, but I'm having some issues. Here's the scenario:
I have some large files, anywhere from 1-10 GB. These are composed of structs, all 128 bytes except the very first one of the file.
I need random access to any one of these structs fast, which from what I read is what a memory-mapped file is perfect for. I also need these memory-mapped files to be able to be opened by other processes, but there won't be any writing, just reading of the file.
I am able to determine the byte offset of the exact struct I need to read.
First I tried making the mem-mapped file like so:
MemoryMappedFile hFileMapping = MemoryMappedFile.CreateFromFile(rampPath, FMode, Path.GetFileNameWithoutExtension(rampPath), GetFileLength(rampPath), access);
However, other processes couldn't open it. So I found that there was another constructor that took in a FileStream. So I changed it to this:
FileStream fs = new FileStream(rampPath, FileMode.OpenOrCreate, FileAccess.Read, FileShare.Read);
MemoryMappedFile hFileMapping = MemoryMappedFile.CreateFromFile(fs, Path.GetFileNameWithoutExtension(rampPath), GetFileLength(rampPath), access, null, HandleInheritability.Inheritable, true);
Other processes could open the mem-mapped file now. Then I try making a view of the mapped file:
MemoryMappedViewAccessor hFileAccessor = hFileMapping.CreateViewAccessor(0, GetFileLength(rampPath), MemoryMappedFileAccess.Read));
This is fine if the files are small, but if they are large I get out of storage and memory exceptions.
So I need to map only a portion of the file at one time, but what is the best way to determine if the struct I'm looking for is in the currently mapped view? Do I keep track of the current mapped view's offset and length and see if the offset of the struct is in there? Is there a better way to go about all this?
Thanks to @Ross Bush in the OP's comments for the suggestions.
What I did was just make the ViewAccessor's capacity a set amount always. And since I have the byte offset I need, I just ran some calculations to determine if the byte offset was inside the ViewAccessor or not. If it wasn't, then change the ViewAccessor to a different offset/capacity so the byte offset fell within it.