I have been looking at some SO questions related to the max size of an array of bytes (here and here) and have been playing with some arrays and getting some results I don't quite understand. My code is as follows:
byte[] myByteArr;
byte[] myByteArr2 = new byte[671084476];
for (int i = 1; i < 2; i++)
{
myByteArr = new byte[671084476];
}
This will compile and upon execution it will throw a 'System.OutOfMemoryException' on the initialization of myByteArr. If I change the 2 in the for loop to a 1 or I comment out one of the initialization's (either myByteArr2 or myByteArr) it will run fine.
Also, byte[] myByteArr = new byte[Int32.MaxValue - 56]; throws the same exception.
Why does this happen when compiled for 32-bit? Aren't they within the 2GB limit?
The limits of a 32-bit program are not per-object. It's a process limit. You cannot have more than 2GB total in use.
Not only that, but in practice, it's often difficult to get anywhere near 2GB due to address space fragmentation. .NET's managed (ie. movable) memory helps somewhat, but doesn't eliminate this problem.
Even if you are using a 64-bit process, you may have a similar problem because in C# arrays are indexed by an int, which is defined as a 32-bit signed integer, and thus can't address past the 2GB boundary in an array of bytes. If you read the answer to the second link carefully, you'll also see that there is a 2GB per object limit. Your array of bytes presumably has some overhead, so it can't get to the full 2GB just for the raw data.
See @Habib's link in the comments for details.