Search code examples
c#listmemory-managementout-of-memorydispose

How to free memory which is allocated by a List in C#


I have a code like this :

var myList = db.Table1.ToList();
/*doing some operations on the list*/
var myList = db.Table2.ToList();
/*again doing some operations on the list*/
var myList = db.Table3.ToList(); // I'm getting out of memory exception here.

I can't retrieve data by pages because I need all of the table once. How can I dispose( I mean free the space which is alloacted by that list) list before I load another table? Thanks.

EDIT :

I'm actually generating many (sometimes thousands of) sublists from myList after I load it. So I really need to learn how to free a list.

EDIT 2 : Here is my full Stacktrace :

   at System.Collections.Generic.List`1.set_Capacity(Int32 value)
   at System.Collections.Generic.List`1.EnsureCapacity(Int32 min)
   at System.Collections.Generic.List`1.Add(T item)
   at System.Data.Entity.Core.Objects.EntityEntry.TakeSnapshot(Boolean onlySnapshotComplexProperties)
   at System.Data.Entity.Core.Objects.Internal.SnapshotChangeTrackingStrategy.TakeSnapshot(EntityEntry entry)
   at System.Data.Entity.Core.Objects.Internal.EntityWrapper`1.TakeSnapshot(EntityEntry entry)
   at System.Data.Entity.Core.Objects.ObjectStateManager.AddEntry(IEntityWrapper wrappedObject, EntityKey passedKey, EntitySet entitySet, String argumentName, Boolean isAdded)
   at System.Data.Entity.Core.Common.Internal.Materialization.Shaper.HandleEntityAppendOnly[TEntity](Func`2 constructEntityDelegate, EntityKey entityKey, EntitySet entitySet)
   at System.Data.Entity.Core.Common.Internal.Materialization.Coordinator`1.ReadNextElement(Shaper shaper)
   at System.Data.Entity.Core.Common.Internal.Materialization.Shaper`1.SimpleEnumerator.MoveNext()
   at System.Data.Entity.Internal.LazyEnumerator`1.MoveNext()
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)
   at WebApplication2.MyMethod in line 2292
   at WebApplication2.Controllers.MyController.MyActtion(String myString) in line 137
   at System.Web.Mvc.ActionMethodDispatcher.Execute(ControllerBase controller, Object[] parameters)
   at System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary`2 parameters)
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary`2 parameters)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<BeginInvokeSynchronousActionMethod>b__39(IAsyncResult asyncResult, ActionInvocation innerInvokeState)
   at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`2.CallEndDelegate(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase`1.End()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethod(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3d()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass46.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3f()

Solution

  • Based on the additional clarifications in the post comments, the db is EF DbContext derived class variable, shared between the 3 calls. And Table1, Table2 and Table3 are DbSet instances inside that context.

    What happens when you use var myList = db.Table1.ToList() is that additionally to reading the table in memory and filling up your list, the EF DbContext also populates the internal DbSet.Local collection with the same objects (so called tracking) in order to be able to detect and apply the changes you made (eventually) to the underlying objects.

    And here is the problem. Event if you Clear, set to null the myList variable, the internal cache (list) still holds all these objects, eventually causing the OutOfMemoryException at some point.

    So the trick is to either totally eliminate the DbContext internal caching by using so called No-Tracking Queries if you are not planning to modify the returned objects and save them back to the database:

    var myList = db.Table1.AsNoTracking().ToList();
    

    or clean up (Dispose and set to null) and use fresh new context (db = new YourDbContext()) for each list process.

    Of course you could combine the above with the other techniques mentioned in the comments for cleaning up your own list variables.