Search code examples
wpflistviewlarge-data

WPF ListView with millions of rows

I need to display a very large amount of data in a list view, up to 5 million or more rows. I am trying to find a solution where I can show all of the 5 million of items without having all of the 5 million items in my applications memory.

So basically the idea is that only a small amount of the items are shown, like 1k or 2k and when scrolling new data is retrieved from a database on demand. So that the application never has more then a couple of thousand item in its application memory. However the user should not realize that this is done, the listview should behave as if it had 5 million rows. Which means if he drags the scrollbar way down it should display the last of the 5 million rows and not the last of the couple of thousand rows that are currently in memory.

Does anyone of you have concept how such a feature can be achieved? I am grateful for any input you can provide.

Thanks, m00ni

Solution

  • Data virtualization in wpf is much discussed - here is a good starting place.

    I have implemented an approach that is somewhat different from anything I've found online. It isn't perfect but it suits my needs quite well.

    I have a generic ItemsProvider interface that can page through data and exposes some other basic functions and info about the backing data. I also created a VirtualizationManager class that inherits from DependencyObject and has dependency properties ScrollableHeight and VerticalOffset. These are bound to the matching properties on a ScrollViewer (found in the templates of your finer ItemsControls). When either of these properties change they fire a callback that calculates how close the scroll viewer is to the end of the (currently loaded) list - for this my VirtualizationManager needs a handle to the ItemsProvider - and if that is less than some minimum amount the ItemsProvider is instructed to load the next page. The whole contraption can be installed on an ItemsControl via a set of attached properties.

    My implementation is rather idiosyncratic, but the idea is fairly simple.

    If you're dealing with very large lists you'll want a solution that not only loads incrementally but also unloads old items. There is at least one such solution at the link above.