Search code examples
c#linqdatatable

Split datatable into multiple fixed sized tables


I have a data table which has 1123 records. I want to split this table into 5 fixed size separate datatables. Size limit for each table is 225.

So size of resulting datatables will be:

DT1 : 225 rows
DT2 : 225 rows
DT3 : 225 rows
DT4 : 225 rows
DT5 : 223 rows (remaining rows)

I was able to find how to split datatable based on the column value using LINQ here.

I also found a way to split datatable into multiple tables here. Wanted to know if there's a better way of doing this. Posting code form the link:

private static List<DataTable> SplitTable(DataTable originalTable, int batchSize)
{
     List<DataTable> tables = new List<DataTable>();
     int i = 0;
     int j = 1;
    DataTable newDt = originalTable.Clone();
   newDt.TableName = "Table_" + j;
   newDt.Clear();
    foreach (DataRow row in originalTable.Rows)
    {
         DataRow newRow = newDt.NewRow();
         newRow.ItemArray = row.ItemArray;
         newDt.Rows.Add(newRow);
         i++;
         if (i == batchSize)
        {
           tables.Add(newDt);
           j++;
          newDt = originalTable.Clone();
          newDt.TableName = "Table_" + j;
          newDt.Clear();
          i = 0;
      }
  }
   return tables;
}

Need help in splitting datatable into fixed size.


Solution

  • I once made this little extension method:

    public static IEnumerable<IEnumerable<T>> ToChunks<T>(this IEnumerable<T> enumerable,
                                                          int chunkSize)
    {
        int itemsReturned = 0;
        var list = enumerable.ToList(); // Prevent multiple execution of IEnumerable.
        int count = list.Count;
        while (itemsReturned < count)
        {
            int currentChunkSize = Math.Min(chunkSize, count - itemsReturned);
            yield return list.GetRange(itemsReturned, currentChunkSize);
            itemsReturned += currentChunkSize;
        }
    }
    

    that cuts any IEnumerable into chunks of the specified chunk size.

    Having this, you can simply do:

    var tables = originalTable.AsEnumerable().ToChunks(225)
                              .Select(rows => rows.CopyToDataTable())
    

    The reason why this could perform better than a straightforward foreach is that list.GetRange is a very efficient method to get a range of rows from a list. I curious to know what you'll find out.