Force deallocate huge memory array

Question

In C# core,

I have a hugh binary data array that I need to use in my code, and I need to free its allocated memory when finish using it.

The code is running in docker for linux (using the base images: microsoft/dotnet:2.1-sdk + microsoft/dotnet:2.1-runtime) - I see the heap usage is always get greater (by htop).

byte[] arr = new byte[1024*1024*1024];
Console.WriteLine("array in gen:{0}", GC.GetGeneration(arr));

// The array is in generation 2.

When I do:

Console.WriteLine("total mem before:{0}", GC.GetTotalMemory(false)); 
GC.Collect();
Console.WriteLine("total mem after:{0}", GC.GetTotalMemory(false)); 

The memory allocated by the array is not disposed, and collect doesn't seems to be work immediately?

I have tried also putting the arr in IDisposable, but can it help?

public class DisposeArr : IDisposable
{
    // Flag: Has Dispose already been called?
    bool disposed = false;
    // Instantiate a SafeHandle instance.
    SafeHandle handle = new SafeFileHandle(IntPtr.Zero, true);

    byte[] Data { get; set; }

    public DisposeArr(long size)
    {
        Data = new byte[size];
    }
    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    // Protected implementation of Dispose pattern.
    protected virtual void Dispose(bool disposing)
    {
        if (disposed)
            return;

        if (disposing)
        {
            Data = null;
            handle.Dispose();
            // Free any other managed objects here.
            //
        }

        disposed = true;
    }
    ~DisposeArr()
    {
      Dispose(false);
    }
}

and in main code:

using (DisposeArr newArr = new DisposeArr(1024 * 1024 * 1024))
{

}
GC.Collect();
GC.WaitForPendingFinalizers();
Console.WriteLine("total mem before:{0}", GC.GetTotalMemory(false)); 
GC.Collect();
Console.WriteLine("total mem after:{0}", GC.GetTotalMemory(false));

How can I force that the memory allocated by the array will disposed immediately after 'GC.Collect'?

Here is the full code. The code may work, but if there is 100%, and 95% (almost Memory) there may be some memory leak (and on docker - in linux - that's may run into memory leak).

    using Microsoft.Win32.SafeHandles;
using System;
using System.Runtime.InteropServices;

namespace TestHeap
{
    public class DisposeArr : IDisposable
    {
        // Flag: Has Dispose already been called?
        bool disposed = false;
        // Instantiate a SafeHandle instance.
        SafeHandle handle = new SafeFileHandle(IntPtr.Zero, true);

        byte[] Data { get; set; }

        public DisposeArr(long size)
        {
            Data = new byte[size];
        }
        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        // Protected implementation of Dispose pattern.
        protected virtual void Dispose(bool disposing)
        {
            if (disposed)
                return;

            if (disposing)
            {
                Data = null;
                handle.Dispose();
                // Free any other managed objects here.
                //
            }

            disposed = true;
        }
        ~DisposeArr()
        {
          Dispose(false);
        }
    }

    class Program
    {
        public void Run()
        {
            for (int i = 0; i < 100; i++)
            {
                Console.WriteLine("total mem:{0}", GC.GetTotalMemory(false)); GC.Collect();
                using (DisposeArr newArr = new DisposeArr(1024 * 1024 * 1024))
                {

                }
                //            byte[] arr = new byte[1024 * 1024 * 1024];
                Console.WriteLine("New. total mem:{0}", GC.GetTotalMemory(false));
                GC.Collect();
                Console.WriteLine("Collect. total mem:{0}", GC.GetTotalMemory(false));
                GC.WaitForPendingFinalizers();
                Console.WriteLine("Pending. total mem:{0}", GC.GetTotalMemory(false)); GC.Collect();
                GC.Collect();
                Console.WriteLine("Collect. total mem:{0}", GC.GetTotalMemory(false));
            }
            Console.ReadLine();
        }
        static void Main(string[] args)
        {
            Program p = new Program();
            p.Run();
        }
    }
}

Thanks.

Answer 1

The code for the answer works, but not if there is a single reference for the array:

i.e, if I write:

byte[] b;
using (DisposeArr newArr = new DisposeArr(1024 * 1024 * 1024))
{
   b = newArr.Data;
}

b = null;

GC.Collect();

The above code doesn't realy dispose the element, because there is a reference to it.

Everything must be refer directly to newArr.data, so there is no byte[] in code at all.

If wee need to refer that object, the only thing I found, is to convert to other class, with unsafe create.

public static class UnsafeConverter
{
    // to do: convert only arrays of the same size short[] <-> ushort[]
    private static unsafe T GetInstanceByPointer<T>(IntPtr pointer)
    {
        try
        {
            var fakeInstance = default(T);

            var sizeInstance = default(int);
            TypedReference typerefSize = __makeref(sizeInstance);


            TypedReference typedReference = __makeref(fakeInstance);
            *(IntPtr*)(&typedReference) = pointer;
            T instance = __refvalue(typedReference, T);
            return instance;
        }
        catch
        {
            return default;
        }
    }
    public static unsafe T GetInstance<T>(object value)
    {
        try
        {
            GCHandle handle = GCHandle.Alloc(value);
            IntPtr px = (IntPtr)handle;
            T instance = GetInstanceByPointer<T>(px);
            handle.Free();

            return instance;
        }
        catch
        {
            return default;
        }            
    }
} 

and in DisposeArr class:

public DisposeArr(byte[] arr)
{
    Data = UnsafeConverter.GetInstance<byte[]>(arr);
}

If I am calling Dispose explicity, I need to keep the array in some collection in code, and add a function that check that collection (whenever there was not any dispose).

i.e:

private static ConcurrentDictionary<int, DisposeArr> _allData = new ConcurrentDictionary<int, DisposeArr>()

The Class declaration:

public class DisposeArr : IDisposable { ...

    public DisposeArr(long size)
    {
        Data = new byte[size];
        _allData.TryAdd(GetHashCode(), this);
    }

    public static CollectAll()
    {
       for (var item in _allData)
       {
          _allData.Value.Dispose();
       }
    }

    protected virtual void Dispose(bool disposing)
    {
        if (disposed)
            return;

        if (disposing)
        {
           ...
           _allData.TryRemove(GetHashCode(), out DisposeArr tmpDisposeArr);
        }
    }

}