Error on recording UTF8 string with Datasnap

Question

Recently i've migrated from D2007 to 10.3.3 and i'm having the issue below.

I have a TClientDataSet that record data an encrypted string into .cds format, by storing it in a TStringField and using the .saveToFile method. Problem is, the string is not being record correclty into the CDS table.

Here my encrypt function :

function encrypt(const ent: string): string;
var m, i, k : integer;
r, s : string;
begin
m := 3;
r := '';
for I := 1 to Length(ent) do
  begin
  k := ord(ent[i]);
  s := chr(k+m);
  r := r + s;
  inc(m);
  end;
result := r;
end;

And here is my decrypt function :

function decrypt(const ent: string): string;
var m, i,j,K : integer;
r, s : string;
begin
m := 3;
r := '';
s := ent;
for I := 1 to length(s) do
  begin
  if ord(s[i]) < 68 then j := 1 else
    j := -1;
  k := ord(s[i]);
  r := r + chr(k-m);
  inc(m);
  end;
result := r;
end;

Now i want to call this function with the string 'engajamento1234'. Here is how the watch shows the encrypted string :

And here is how the string is actually inserted into the TStringField column of the table :

Because the string end up being insert incorrectly on the clientdataset, when i call decrypt function with 'hrlgqivoy?|?ACE' string, i get the incorrect original string as 'engajamen3o1234'...

Seems like this problem is related to UTF8 support in the new Delphi version, which the old version did not have.

What i need to do to the string record correctly into the clientdataset and the .cds file, so i get the correct decrypted string back ?

Answer 1

I've expanded your code into an MRE which illustrates the problem you are having and how to solve it. Basically it seems that the string field you are using to store the encrypted data is a TStringField, whereas it needs to be a TWideStringField to work with Unicode. TStringField's SetAsString and GetAsString methods both use calls which treat the data as an ANSIString (which is the type of Delphi string which predates the introduction of Unicode).

The code uses a conditional define, UseWideString, which determines whether the CDS field is of type ftString (as was used in D2007) or ftWideString, which is the Unicode equivalent. You should find that with the UseWideString define in effect, the code executes without error, meaning that the decrypted string is the same as the original input. If you comment out the UseWideString define, the

Assert(sInput = sDecrypted);

fails.

Code

program EncryptTest;

{$APPTYPE CONSOLE}

uses
  SysUtils, db, dbclient;

function encrypt(const ent: string): string;
var m, i, k : integer;
r, s : string;
begin
m := 3;
r := '';
for I := 1 to Length(ent) do
  begin
  k := ord(ent[i]);
  s := chr(k+m);
  r := r + s;
  inc(m);
  end;
result := r;
end;

function decrypt(const ent: string): string;
var m, i,j,K : integer;
r, s : string;
begin
m := 3;
r := '';
s := ent;
for I := 1 to length(s) do
  begin
  if ord(s[i]) < 68 then j := 1 else
    j := -1;
  k := ord(s[i]);
  r := r + chr(k-m);
  inc(m);
  end;
result := r;
end;

var
  sInput,
  sEncrypted,
  sDecrypted : String;
  CDS : TClientDataSet;
  Field : TField;

begin
  CDS := TClientDataSet.Create(Nil);

{.$define UseWideString}
{$ifdef UseWideString}
  Field := TWideStringfield.Create(Nil);
{$else}
  Field := TStringfield.Create(Nil);
{$endif}
  Field.FieldKind := fkData;
  Field.Size := 80;
  Field.FieldName := 'Something';
  Field.DataSet := CDS;

  CDS.CreateDataSet;
  CDS.Append;
  CDS.Edit;

  sInput := 'engajamento1234';
  sEncrypted := encrypt(sInput);
  Field.AsString := sEncrypted;
  sDecrypted := decrypt(Field.AsString);
  Assert(sInput = sDecrypted);

  CDS.Cancel;
  Field.Free;
  CDS.Free;

end.

Btw, in your decrypt function, the

if ord(s[i]) < 68 then j := 1 else
  j := -1;

is superfluous because the value of j is never used.