Assign, memcpy or other method required for TClientDataset or TDataSetProvider (dbexpress)

Question

I'm usign the dbExpress components within the Embarcadero C++Builder XE environment. I have a relatively large table with something between 20k and 100k records, which I display in a DBGrid. I am using a DataSetProvider, which is connected to a SQLQuery and a ClientDataSet, which is connected to the DataSetProvider.

I also need to analyze the data and therefore I need to run through the whole table. For smaller tables I always used code, which is basically something like this:

    Form1->ClientDataSet1->First();
    while(!Form1->ClientDataSet1->Eof){
        temp=Form1->ClientDataSet1->FieldByName("FailReason")->AsLargeInt;
       //do something with temp
       Form1->ClientDataSet1->Next();
       }

Of course this works out, but it is very slow, when I need to run through the whole DBGrid. For some 50000 records in can take up to some minutes. My suspicion is that the most perform is lost since the DBGrid needs to be repainted as the actual Dataset increments its address. Therefore I am looking for a method which allows me to either read the data without manipulating the actual ClientDataSet. Maybe a method which copies the data of a column into a variable, or another way to run through the datasets, which is more efficient. I am sure if I would have a copy in a variable the operation would take less than a few seconds... I googled now for hours, but didn't find anything useful so far.

Best regards, Bodo

Answer 1

if your cds is connected to some db-aware control(-s) (via TDataSource) then first of all consider using DisableControls()

another option would be to avoid utilizing FieldByName within the loop