Search code examples
c#gcallowverylargeobjectslarge-data

Extra large array

I have to operate with two-dimensional array(s) with size 34000x34000 items for math calculations.

Obvious problem - is impossibility of CLR to store such big parts of data in memory. I tried to use MemoryMappedFile, but it fails too while I'm trying to create viewer for the object: MemoryMappedFile.CreateViewAccessor(). Are there any other already existed ways to store large arrays? ('cause I havent' much time for trying to implement custom large data storage)

thanks

Solution

  • The <gcAllowVeryLargeObjects> configuration element allows for over-2-Gb-arrays for 64-bit processes. You might give that a try.

    Also, see if sparse arrays can help you:

    A sparse array is an array in which most of the elements have the default value (usually 0 or null). The occurrence of zero-value elements in a large array is inefficient for both computation and storage.