TDataSet EOF loop logic doesn't seem to match what is documented

Question

I found Using the Eof and Bof Properties when searching on how to loop through a TDataSet. However, I find that the Eof becomes true when the last record is retrieved via Next() instead of after retrieval of a non-existent record, as documented. The DB is SQLite using FireDAC.

Basically, this is missing the last record:

dataset->Filter = _D("MyFieldToSetDefaultValue is NULL");
dataset->Filtered = true;

dataset->First();
while (!dataset->Eof) {
  dataset->Edit();
  // ...
  dataset->Post();
  dataset->Next();
}
// oops - if there were 3 records the above loop only handled 2 of them, if I check
// the record returned after Next() it is the last record, only Eof is now true?

This works (even if only 1 record exists):

dataset->Filter = _D("MyFieldToSetDefaultValue is NULL");
dataset->Filtered = true;

dataset->First();
bool eof = dataset->Eof;
while (!eof) {
  dataset->Edit();
  // ...
  dataset->Post();
  eof = dataset->Eof;
  dataset->Next();
}

Update 1:

Okay, so this gets more interesting. This works:

dataset->Filter = _D("MyFieldToSetDefaultValue is NULL");
dataset->Filtered = true;

dataset->First();
while (!dataset->Eof) {
  dataset->Next();
}

One thing I noticed is the order of processing the records was different with the above. The above came in alphabetic order, which is what the grid is currently sorted in. This process runs after an import routine which added several records, so it's reporting Eof on the last record in the order it would be in the grid, but when using Edit() / Post() it's in some other order. In other words, it added 3 items:

New eAcc Net

When it processed them in the loop that doesn't work, it was in order of:

eAcc New Net

It was after processing the New record that Eof reported true, which it would be once sorted.

So, maybe there is something I need to do before disabling controls, or put in a delay? Has anyone else come across this? How do you handle it?

Update 2:

If I just set dataset->Filtered = false, it works as expected. I would rather use the filter than process each record and check the fields that way.

It appears to me that maybe because the condition that I fix no longer exists, it gets confused on what the Eof is since the filter excluded it? Maybe just keep calling First() over and over?

Update 3:

Yes, it seems this works, but is there a better way?

dataset->Filter = _D("MyFieldToSetDefaultValue is NULL");
dataset->Filtered = true;

dataset->First();
while (!dataset->Eof) {
  dataset->Edit();
  // ...
  dataset->Post();
  dataset->First(); // call first again instead of next since updated record no longer in filter.
}

Answer 1

If the act of editing the record is removing the record from the current filter, then presumably the next record in the DataSet will become the new active record when the edited record is removed. This is stated in the documentation for the Filtered property:

Note: When filtering is enabled, user edits to a record may mean that the record no longer meets a filter's test condition. The next time an attempt is made to the read the record from the dataset while the filter is in effect, the record is not visible. If that happens, the next record that passes the filter condition becomes the active record.

So, if every record is being removed when edited, then try simply not seeking the DataSet at all inside the loop, eg:

dataset->Filter = _D("MyFieldToSetDefaultValue is NULL");
dataset->Filtered = true;

dataset->First();
while (!dataset->Eof) {
  dataset->Edit();
  // ...
  dataset->Post(); // <-- active record is removed from
  // the filter, making the next record the active record,
  // so do not call First() or Next() here!
}