Force a DLL to be loaded above 2GB (0x80000000) in a 32-bit process on Windows

Question

To test a corner case in our debugger, I need to come up with a program which has a DLL loaded above 2GB (0x80000000). Current test case is a multi-GB game which loads >700 DLLs, and I'd like to have something simpler and smaller. Is there a way to achieve it reliably without too much fiddling? I assume I need to use /LARGEADDRESSAWARE and somehow consume enough of the VA space to bump the new DLLs above 2GB but I'm fuzzy on the details...

Answer 1

Okay, it took me a few tries but I managed to come up with something working.

// cl /MT /Ox test.cpp /link /LARGEADDRESSAWARE
// occupy the 2 gigabytes!
#define ALLOCSIZE (64*1024)
#define TWOGB (2*1024ull*1024*1024)

#include <windows.h>
#include <stdio.h>

int main()
{
  int nallocs = TWOGB/ALLOCSIZE;
  for ( int i = 0; i < nallocs+200; i++ )
  {
   void * p = VirtualAlloc(NULL, ALLOCSIZE, MEM_RESERVE, PAGE_NOACCESS);
   if ( i%100 == 0)
   {
     if ( p != NULL )
       printf("%d: %p\n", i, p);
     else
     {
       printf("%d: failed!\n", i);
       break;
     }
   }
  }
  printf("finished VirtualAlloc. Loading  a DLL.\n");
  //getchar();
  HMODULE hDll = LoadLibrary("winhttp");

  printf("DLL base: %p.\n", hDll);
  //getchar();
  FreeLibrary(hDll);
}

On Win10 x64 produces:

0: 00D80000
100: 03950000
200: 03F90000
[...]
31800: 7FBC0000
31900: 00220000
32000: 00860000
32100: 80140000
32200: 80780000
32300: 80DC0000
32400: 81400000
32500: 81A40000
32600: 82080000
32700: 826C0000
32800: 82D00000

32900: 83340000
finished VirtualAlloc. Loading  a DLL.
DLL base: 83780000.