Search code examples
c#arraysmemory-managementilist

C# chunked array

I need to allocate very large arrays of simple structs (1 GB RAM). After a few allocations/deallocations the memory becomes fragmented and an OutOfMemory exception is thrown.

This is under 32 bit. I'd rather not use 64 bit due to the performance penalty I get - the same application runs 30% slower in 64 bit mode.

Are you aware of some implementations of IList compatible arrays which allocate memory in chunks and not all at once? That would avoid my memory fragmentation problem.

Solution

  • Josh Williams presented a BigArray<T> class on his blog using a chunked array:

    BigArray<T>, getting around the 2GB array size limit

    You will find more useful information in this related question:

    C# huge size 2-dim arrays

    A simple ad-hoc fix might be to enable the 3GB switch for your application. Doing so allows your application to use more than the 2GB per-process limit of 32-bit Windows. However, be aware that the maximum object size that the CLR allows is still 2GB. The switch can be enabled using a post-built action for your main executable:

    call "$(DevEnvDir)..\tools\vsvars32.bat"
editbin.exe /LARGEADDRESSAWARE "$(TargetPath)"