System.OutOfMemoryException Entity Framework 6 for db.SaveChanges

Question

I'm running a batch job of inserting events into a database with entity framework. Total size varies but around 3.5 million events are normal. This list is split up into groups of 60-500k events that are then added to the database.

However when around one million events have been added I get System.OutOfMemoryException and have to start over. So to complete the program it has to be run 3-4 times now. How can I clear my db object after SaveChanges() has been called so this won't happen?

public DbSet<Events> Events { get; set; }
...
var groupedList = list.GroupBy(x => x.Import.FileName);
var db = new ApplicationDbContext();

foreach (var group in groupedList)
{
   db.Events.AddRange(group);
   db.SaveChanges();
}

Stack trace:

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Data.Entity.Core.Mapping.Update.Internal.KeyManager.GetPrincipals(Int32 identifier)
at System.Data.Entity.Core.Mapping.Update.Internal.DynamicUpdateCommand..ctor(TableChangeProcessor processor, UpdateTranslator translator, ModificationOperator modificationOperator, PropagatorResult originalValues, PropagatorResult currentValues, DbModificationCommandTree tree, Dictionary`2 outputIdentifiers)
at System.Data.Entity.Core.Mapping.Update.Internal.UpdateCompiler.BuildInsertCommand(PropagatorResult newRow, TableChangeProcessor processor)
at System.Data.Entity.Core.Mapping.Update.Internal.TableChangeProcessor.CompileCommands(ChangeNode changeNode, UpdateCompiler compiler)
at System.Data.Entity.Core.Mapping.Update.Internal.UpdateTranslator.<ProduceDynamicCommands>d__a.MoveNext()
at System.Linq.Enumerable.<ConcatIterator>d__58`1.MoveNext()
at System.Data.Entity.Core.Mapping.Update.Internal.UpdateCommandOrderer..ctor(IEnumerable`1 commands, UpdateTranslator translator)
at System.Data.Entity.Core.Mapping.Update.Internal.UpdateTranslator.ProduceCommands()
at System.Data.Entity.Core.Mapping.Update.Internal.UpdateTranslator.Update()
at System.Data.Entity.Core.EntityClient.Internal.EntityAdapter.<Update>b__2(UpdateTranslator ut)
at System.Data.Entity.Core.EntityClient.Internal.EntityAdapter.Update[T](T noChangesResult, Func`2 updateFunction)
at System.Data.Entity.Core.EntityClient.Internal.EntityAdapter.Update()
at System.Data.Entity.Core.Objects.ObjectContext.<SaveChangesToStore>b__35()
at System.Data.Entity.Core.Objects.ObjectContext.ExecuteInTransaction[T](Func`1 func, IDbExecutionStrategy executionStrategy, Boolean startLocalTransaction, Boolean releaseConnectionOnSuccess)
at System.Data.Entity.Core.Objects.ObjectContext.SaveChangesToStore(SaveOptions options, IDbExecutionStrategy executionStrategy, Boolean startLocalTransaction)
at System.Data.Entity.Core.Objects.ObjectContext.<>c__DisplayClass2a.<SaveChangesInternal>b__27()
at System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.Execute[TResult](Func`1 operation)
at System.Data.Entity.Core.Objects.ObjectContext.SaveChangesInternal(SaveOptions options, Boolean executeInExistingTransaction)
at System.Data.Entity.Core.Objects.ObjectContext.SaveChanges(SaveOptions options)
at System.Data.Entity.Internal.InternalContext.SaveChanges()
at System.Data.Entity.Internal.LazyInternalContext.SaveChanges()
at System.Data.Entity.DbContext.SaveChanges()
at Project.Program.Main(String[] args) in C:\Users\oscar\Documents\Solution\Project\Program.cs:line 135

Answer 1

After inserting one batch you should recreate your DbContext to clear the ChangeTracker. This should not only fix your memory issue, but also make things a bit faster. Something like

foreach (var group in groupedList)
{
   using(var db = new MyDbContext()){
     db.Events.AddRange(group);
     db.SaveChanges();
   }
}