Is it suggestable to use generics for large amount of data?

Question

I'm having let's say thousands of Customer records and I have to show them on a webform. Also, I have one CustomerEntity which has 10 properties. So when I fetch data in using a DataReader and convert it into List<CustomerEntity> I am required to loop through the data two times.

So is the use of generics suggestable in such a scenario? If yes then what will be my applications performance?

For E.g.

In CustomerEntity class, i'm having CustomerId & CustomerName propeties. And i'm getting 100 records from Customer Table Then for Preparing List i've wrote following code

while (dr.Read()) 
    { 
       // creation of new object of customerEntity 
       // code for getting properties of CustomerEntity 
       for (var index = 0; index <  MyProperties.Count; index++) 
       { 
         MyProperty.setValue(CustEntityObject,dr.GetValue(index)); 
        } 
      //adding CustEntity object to List<CustomerEntity> 
    }

How can i avoid these two loops. Is their any other mechanism?

Answer 1

I'm not really sure how generics ties into data-volume; they are unrelated concepts... it also isn't clear to me why this requires you to read everything twice. But yes: generics are fine when used in volume (why wouldn't they be?). But of course, the best way to find a problem is profiling (either server performance or bandwidth - perhaps more the latter in this case).

Of course the better approach is: don't show thousands of records on a web form; what is the user going to do with that? Use paging, searching, filtering, ajax, etc - every trick imaginable - but don't send thousands of records to the client.

---

Re the updated question; the loop for setting properties isn't necessarily bad. This is an entirely appropriate inner loop. Before doing anything, profile to see if this is actually a problem. I suspect that sheer bandwidth (between server and client, or server and database) is the biggest issue. If you can prove that this loop is a problem there are things you can do do optimise:

• switch to using PropertyDescriptor (rather than PropertyInfo), and use HyperDescriptor to make it a lot faster
• write code with DynamicMethod to do the job - requires some understanding of IL, but very fast
• write a .NET 3.5 / LINQ Expression to do the same and use .Compile() - like the second point, but (IMO) a bit easier

I can add examples for the first and third bullets; I don't really want to write an example for the second, simply because I wouldn't write that code myself that way any more (I'd use the 3rd option where available, else the 1st).