Populate ASP.NET ListBox asynchronously without blocking UI

Question

I have a list box in an ASP.NET user control that I want to populate with a potentially large List<string> retrieved from the database (up to 50k items). As the list box is being populated (in the page load event), I don't want to block the UI. Using synchronous code the page isn't displayed until the list box population is completed:

List<string> items = GetItemsFromDB();    
foreach (var item in items) // can take some time, blocks UI
{
    if (shouldItemBeListed(item))
        ListBox1.Items.Add(item);
}

I come from desktop apps background where achieving this is simple using e.g. BackgroundWorker - the user will see the list box populated with new items in real-time. Web stuff seems to be a bit more complex and I'd like some advice on what would be the best way to go about this.

Here's what I tried:

1. Creating a background thread. Doesn't work very well - everything is still pretty unresponsive compared to the desktop (WinForms) version, and the list is not updated/refreshed as expected:

Thread thread = new Thread(new ThreadStart(Populate));
thread.IsBackground = true;
thread.Start(); 

private void Populate()
{
    List<string> items = GetItemsFromDB();    
    foreach (var item in items)
    {
        if (shouldItemBeListed(item))
            ListBox1.Items.Add(item);
    }
}

2. BackgroundWorker thread. Works a bit better then 1. (at least the list box is being updated with new items), but still very unresponsive and slow.

var bw = new BackgroundWorker();
bw.DoWork += (o, args) => Populate();
bw.RunWorkerAsync();

I can always break the list box items into smaller groups and populate more as requested by user, but how is this usually implemented properly?

Answer 1

You can't use this approach, because of the asp.net lifecycle (and the web in general). Basically, each request to the webserver will spawn you asp.net page class, generate the html, send it to the client, and close the instance. Using multithreading technique won't help, because you will change the control tree AFTER the server has sent the answer.

To solve your issue, you should use an <asp:hidden runat='server'> field.

Then, using some kind of client side script, dynamically populate a drop down list from a web service call.

Something like this using jQuery (from memory):

<select id='ddl' onchange='document.getElementById("<%= hidValue.ClientID %>").value=this.value'>
    <option value=''>Loading...</option>
</select>
<asp:Hidden runat='server' id='hidValue' />

$(function(){
   $.ajax("... your api endpoit").then(function(result){
       var items = result.items; // actual implementation will change
       var ddl = $(ddl);
       ddl.html(''); // remove the 'loading' option.
       foreach(var item in items) {
           var option = $("<option>");
           option.attr("value",item.value);
           option.text(item.text);
           ddl.append(option);
       }    
   });
});

The reason not to use an <asp:DropDownList> is that you can't change its children between postbacks. So we use a pure client one, and sync changes in an hidden control, which can be read on postback.