Delphi: write/read variables and records to/from file

Question

Working on my project in XE8, i've faced a necessity to save and read custom project files, which store variables and records of different types. Initially, my approach to solving that problem seemed to work, but in actual project it proved faulty.

My method for creating a file, storing a "Categories" record:

var 
SavingStream: TFileStream;
         i,j: Integer;
begin
SavingStream:=TFileStream.Create('SAVE.test', fmCreate or fmOpenWrite or fmShareDenyWrite);
SavingStream.Position:=0;
i:=Length(Categories);                 **// storing size of an array in a temp variable**
SavingStream.WriteBuffer(i,SizeOf(i)); **// for some reason i couldn't save it directly**
for i:=0 to Length(Categories)-1 do
   begin         
   **{ String }**
   SavingStream.WriteBuffer(Categories[i].Name,SizeOf(Categories[i].Name));  
   **{ Integer }**   
   SavingStream.WriteBuffer(Categories[i].ID,SizeOf(Categories[i].ID));
   **{ Boolean }**
   SavingStream.WriteBuffer(Categories[i].Default,SizeOf(Categories[i].Default))
   **{ Same routine for dynamic array }**
   j:=Length(Categories[i].ChildrenType);
   SavingStream.WriteBuffer(j,SizeOf(j));
   if j>=1 then for j:=0 to Length(Categories[i].ChildrenType)-1 do SavingStream.WriteBuffer(Categories[i].ChildrenType[j],SizeOf(Categories[i].ChildrenType[j]));
   end;
end;

And then reading it:

var 
SavingStream: TFileStream;
         i,j: Integer;
begin
try
SavingStream.ReadBuffer(i,SizeOf(i));
SetLength(Categories,i);
for i:=0 to Length(Categories)-1 do
   begin
   SavingStream.ReadBuffer(Categories[i].Name,SizeOf(Categories[i].Name));     
   SavingStream.ReadBuffer(Categories[i].ID,SizeOf(Categories[i].ID));         
   SavingStream.ReadBuffer(Categories[i].Default,SizeOf(Categories[i].Default));
   SavingStream.ReadBuffer(j,SizeOf(j));
   SetLength(Categories[i].ChildrenType,j);
   if j>=1 then for j:=0 to Length(Categories[i].ChildrenType)-1 do SavingStream.ReadBuffer(Categories[i].ChildrenType[j],SizeOf(Categories[i].ChildrenType[j]));
   end;
finally
SavingStream.Free;
end;

One of the main problems is that i don't entirely understand the logic behind this method. It is my understanding, that SizeOf(i) is basically saying to take a certain part of an otherwise homogeneous file and taking it as a variable's value. But how do i store strings and arrays with a variable size? I know it's possible to limit it's size in the record itself, but there are certain string variables that i don't want to be limited in.

Thus i need your advice whether the method i'm using is any good and how can i make it work in my specific case. Maybe there is a better way to store this information? Keep in mind, that i have to store a vast range of different types, including images.

Thx in advance.

Answer 1

You need to serialize variable-length data, like strings, into a flat format that does not contain any pointers to other memory addresses.

Try something like this:

procedure WriteIntegerToStream(Stream: TStream; Value: Integer);
begin
  Stream.WriteBuffer(Value, Sizeof(Value));
end;

procedure WriteBooleanToStream(Stream: TStream; Value: Boolean);
begin
  Stream.WriteBuffer(Value, Sizeof(Value));
end;

procedure WriteStringToStream(Stream: TStream; const Value: String);
var
  S: UTF8String;
  Len: Integer;
begin
  S := UTF8String(Value);
  Len := Length(S);
  WriteIntegerToStream(Stream, Len);
  Stream.WriteBuffer(PAnsiChar(S)^, Len);
end;

var 
  SavingStream: TFileStream;
  i, j: Integer;
begin
  SavingStream := TFileStream.Create('SAVE.test', fmCreate or fmOpenWrite or fmShareDenyWrite);
  try
    WriteIntegerToStream(SavingStream, Length(Categories));
    for i := 0 to Length(Categories)-1 do
    begin         
      WriteStringToStream(SavingStream, Categories[i].Name);
      WriteIntegerToStream(SavingStream, Categories[i].ID);
      WriteBooleanToStream(SavingStream, Categories[i].Default);
      WriteIntegerToStream(SavingStream, Length(Categories[i].ChildrenType));
      for j := 0 to Length(Categories[i].ChildrenType)-1 do
      begin
        // write ChildrenType[j] data to SavingStream as needed...
      end;
  finally
    SavingStream.Free;
  end;
end;

Then you can do something similar when reading the file back:

function ReadIntegerFromStream(Stream: TStream): Integer;
begin
  Stream.ReadBuffer(Result, Sizeof(Result));
end;

function ReadBooleanFromStream(Stream: TStream): Boolean;
begin
  Stream.ReadBuffer(Result, Sizeof(Result));
end;

function ReadStringFromStream(Stream: TStream): String;
var
  S: UTF8String;
  Len: Integer;
begin
  Len := ReadIntegerFromStream(Stream);
  SetLength(S, Len);
  Stream.ReadBuffer(PAnsiChar(S)^, Len);
  Result := String(S);
end;

var 
  LoadingStream: TFileStream;
  i, j: Integer;
begin
  LoadingStream := TFileStream.Create('SAVE.test', fmOpenRead or fmShareDenyWrite);
  try
    i := ReadIntegerFromStream(LoadingStream);
    SetLength(Categories, i);
    for i := 0 to Length(Categories)-1 do
    begin
      Categories[i].Name := ReadStringFromStream(LoadingStream);
      Categories[i].ID := ReadIntegerFromStream(LoadingStream);
      Categories[i].Default := ReadBooleanFromStream(LoadingStream);
      j := ReadIntegerFromStream(LoadingStream);
      SetLength(Categories[i].ChildrenType, j);
      for j := 0 to Length(Categories[i].ChildrenType)-1 do
      begin
        // read ChildrenType[j] data from LoadingStream as needed...
      end;
    end;
  finally
    LoadingStream.Free;
  end;
end;