Large query how to return results back in sections C#/ASP.NET MVC 4

Question

I have an application written in ASP.NET MVC 4. I have a requirement to return large results from a table accessed with oledbdatareader.

I am using AJAX to return a JsonResult object that contains a List: List<TableRow>

What I do not understand is, if I am in the DataReader loop

using (OleDbDataReader reader = command.ExecuteReader())
{
   while (reader.Read())
   {
       names.Add(Convert.ToString(reader[0]));
   }
}

Is there a way to periodically send the list object, and then create a new object to pickup and continue on?

Answer 1

Technically the server can only return a single response to every request, so there is not way to do what you want (short of setting up some sort of crazy socket stuff).

I'd flip what you're doing on its head and have your javascript request chunks of the dataset in batches of 1000 (or whatever size), and have it start rendering while requesting the next chunk.

Better yet, you could implement some form of infinite scrolling in your UI so that the next chunk is only requested just in time for it to be displayed, that way you're not sending unneeded data to the client.