Going out of memory

Question

I am using WCF data service to get data from server within my winforms application. I am trying to show bulky report having millions of records. Although i am fetching data page wise and storing into a collection but going out of memory.

Here is the code used to find total records and allocate memory for List.

int totalRecords = ReportingService.Instance.CountRecords_ReportItemWiseSell(d1, d2);
List<Report_ItemWiseSellEntity> reportItems = new List<Report_ItemWiseSellEntity>(totalRecords);

Here is the code to collect all paged data

int totalPageCount = (totalRecords / pageSize) + 1;
lvReport.Items.Clear();
for (int i = 1; i <= totalPageCount; i++){
   var tmpItems = new List<Report_ItemWiseSellEntity>();
   tmpItems = ReportingService.Instance.GetItemWiseSellReport(d1, d2, i, pageSize);
   reportItems.AddRange(tmpItems);

   ... //other stuff
   tmpItems = null;
   Application.DoEvents();
}

Can anybody suggest how to overcome this memory issue. Is there any other alternative? Thanks for sharing your wisdom and time.

Answer 1

Unless ... // other stuff contains something crucial to the issue, your fetching of data page-wise isn't helping with memory consumption on the WinForms side, since you are cumulatively adding all the instances of Report_ItemWiseSellEntity retrieved from the service into the reportItems list.

Since your UI is never going to be able to display "millions of records" at the same time, your strategy should be to hold in memory in the WinForms application only those records which are currently being displayed/processed at any time. You will need to rethink your design for binding data to the UI and coordinate retrieving of "pages" of data with the user's navigation of the report through the UI.