Search code examples
c#entity-framework-6

c# EF6 asynchronous saving/editing


I am trying to save roughly 20,000 records and it takes an unforgivable amount of time to finish. What is the more optimal way of achieving this?

Here's what I currently have:

public async Task SaveOrUpdateItemAsync(List<DepartmentalizedItem> departmentalizedItems)
{
    using(WarehouseContext dbContext = new WarehouseContext())
    {
        using(SemaphoreSlim throttler = new SemaphoreSlim(20))
        {
            var tasks = departmentalizedItems.Select(async item =>
            {
                await throttler.WaitAsync();
                if (item.PK_DepartmentalizedItemId == 0)
                    dbContext.DepartmentalizedItems.Add(item);
                else
                {
                    var deptItem = await dbContext.DepartmentalizedItems.FindAsync(item.PK_DepartmentalizedItemId);
                    dbContext.Entry(deptItem).CurrentValues.SetValues(item);
                }
                throttler.Release();
            });

            await Task.WhenAll(tasks);
        }
        await dbContext.SaveChangesAsync();
    }
}

I've also tried Parallel.ForEach but I'm having synchronization issues.

Thank you.


Solution

  • Thank you for all answers, comments and suggestion. All of those has been taken into consideration and I have finally arrived to a more optimal way of saving/editing 20,000 records in bulk. As suggested by Hasse, I took a peek on BulkInsert, but this is a paid library, thus I tried to find any alternatives and fortunately I found N.EntityFramework.Extensions.

    Here's what solved my issue:

    public async Task BulkSaveAndUpdateAsync(List<DepartmentalizedItem> departmentalizedItems, CancellationToken cancellationToken)
        {
            using (WarehouseContext dbContext = new WarehouseContext())
            {
                var toAddItems = departmentalizedItems.Where(i => i.PK_DepartmentalizedItemId == 0);
                var toUpdateItems = departmentalizedItems.Where(i => i.PK_DepartmentalizedItemId > 0);
                await dbContext.BulkInsertAsync(toAddItems, cancellationToken);
                await dbContext.BulkUpdateAsync(toUpdateItems, cancellationToken);
            }
        }