Delphi ClientDataSet - How to add new data fields to existing dataset?

I'm having trouble working out how to add new data fields to an old dataset file. For example, an old dataset might have an ID field only. Later on we decide we need an ISACTIVE field. I want to reopen my ID-only data and then resave it with ISACTIVE values added. For example:

CDS := TClientDataset.Create(nil);
with TIntegerField.Create(CDS) do
begin
  FieldName := 'ID';
  FieldKind := fkData;
  DataSet := CDS;
end;
CDS.CreateDataSet;

CDS.Close;
with TBooleanField.Create(CDS) do
begin
  FieldName := 'ISACTIVE';
  FieldKind := fkData;
  DataSet := CDS;
end;
CDS.Open; // <--Raises EDatabaseError with message 'Field 'ISACTIVE' not found'.

I had a look for similar questions, the closest I found was one that concerned adding new calculated fields to a dataset only. This method above works fine for adding a calculated field.

At the moment the only (messy) solution I can think of is to load the ID-only data into a temporary dataset, then create a new dataset with ID and ISACTIVE fields defined, then loop through the ID-only dataset and copy the records across to the new dataset.

Solution

There is a simple way to do this.

If you have a CDS with an Integer ID Field and a String 80 Name field, and you save the dataset to XML, as in

AFileName := 'C:\Temp\CDSData.Xml';
CDS1.SaveToFile(AFileName, dfXML);

the resulting XML file will look like this (for D7)

<?xml version="1.0" standalone="yes"?>  
<DATAPACKET Version="2.0">
  <METADATA>
    <FIELDS>
      <FIELD attrname="ID" fieldtype="i4"/>
      <FIELD attrname="Name" fieldtype="string" WIDTH="80"/>
    </FIELDS><PARAMS CHANGE_LOG="1 0 4"/>
  </METADATA>
  <ROWDATA>
    <ROW RowState="4" ID="1" Name="one"/>
  </ROWDATA>
</DATAPACKET>

You can then use MSXML or your favourite XML processor to make the trivial change to add additional FIELD node(s) to the METADATA defining the CCDS's datapacket, to add the extra field(s). You then reload the CDS from the XML. The added field(s) values will be NULL of course and for this technique to work, you must not have persistent TFields defined on the CDS at the time you reload it from the saved XML.

Example code:

procedure TForm1.CopyWithAddedFields;
var
  SS : TStringStream;
  XMLDoc : IXmlDomDocument;
  FieldsNode : IXmlDomNode;
  FieldElement : IXmlDomElement;
begin
  SS := TStringStream.Create('');
  try
    //  Save the CDS's current contents in XML format, close it and clear any presistent fields
    CDS1.SaveToStream(SS, dfXML);
    CDS1.Close;
    CDS1.Fields.Clear;

    //  Next create an XML Document object and load the saved dataset into it
    XMLDoc := CoDomDocument.Create;
    XMLDoc.LoadXML(SS.DataString);

    //  Find the FIELDS node and add a new FIELD node to it       
    FieldsNode := XMLDoc.selectSingleNode('/DATAPACKET/METADATA/FIELDS');
    FieldElement := XMLDoc.createElement('FIELD');
    FieldElement.SetAttribute('attrname', 'Active');
    FieldElement.SetAttribute('fieldtype', 'boolean');
    FieldsNode.appendChild(FieldElement);

    // Save the XML to the stream
    SS.Size := 0;
    SS.WriteString(XmlDoc.xml);
    SS.Position := 0;

    //  Reload the ClientDataset
    CDS1.LoadFromStream(SS);
  finally
    XMLDoc.Free;
    SS.Free;
  end;
end;

Obviously, you could load the modified XML into a different CDS instead if you wanted to.

Of course, you could even add the extra FIELD nodes to the XML just by loading it into a TStringList if you were prepared to do a certain amount of string-twiddling yourself.

Fwiw, I stumbled upon this ruse when trying to modify the XML of a CDS to include extra information for each ROW node in the XML file; it turned out that the LoadFromFile & LoadFromStream processes were completely oblivious to the information I added.