Search code examples
c++memory-mapped-filesmemory-mapping

Memory mapping a huge file in 32 bit software running on 64 bit OS


I have performance problems with reading huge files. I would like to use memory mapping to solve the problems. Input files are up to 10 GB of size, my program (written in C++) is built with 32 bit configuration, the target system is Windows 64 bit and has 24 GB of RAM. Is this possible to map whole a huge file to memory? If yes, how can my program access to the "high" address area (> 0xFFFFFFFF, theoritically, because my program is 32 bit, so pointers are 32 bit also)?


Solution

  • You can't. A 32-bit program uses 32-bit pointers that can't go past 4GB, even when running on a 64-bit OS.

    One thing that may help a little, though, is to link your program with the /LARGEADDRESSAWARE option. By default, 32-bit Windows programs are only able to use 2GB of the address space, because some programs assume that a pointer's uppermost bit is always zero and use it as a flag to store additional information. That'd break if Windows started allocating memory above 2GB, so you have to link your program with a special option that tells Windows your program doesn't abuse that bit. This lets your program use the full 4GB of address space instead of being limited to 2GB.