Is this economical?

Question

Just wanting to see if there is a better way to do the following(there is always a better way for everything) because it does delay the application when loading due the amount of data.

I want to fill an array of records with data I have stored in csv file, I currently have it fixed length for the array but will later make it dynamic so I can add to the csv file.

    type
          TStarCoords = Packed record
            szSystem: String[40];
            fCoordX: Single;
            fCoordY: Single;
            fCoordZ: Single;
          end;

    SystemCoords: Array [0 .. 22379] of TStarCoords;

Const
SYSTEMS = 'Data\Systems.csv';

I then fill the array on the oncreate event

procedure TForm1.FormCreate(Sender: TObject);
var
  szFile, sRecord: string;
  Row, Index, i: Integer;
  slList: TStringList;
begin

  szFile := ExtractFilePath(ParamStr(0)) + SYSTEMS;

  if FileExists(szFile) then
    try
      slList := TStringList.Create;
      slList.LoadFromFile(szFile);

      for Row := 0 to slList.Count - 1 do
      begin
        sRecord := slList[Row];

        index := Pos(',', sRecord);
        if index > 0 then
        begin
          SystemCoords[Row].szSystem := Copy(sRecord, 1, index - 1);
          Delete(sRecord, 1, index);
        end;

        index := Pos(',', sRecord);
        if index > 0 then
        begin
          SystemCoords[Row].fCoordX := StrToFloat(Copy(sRecord, 1, index - 1));
          Delete(sRecord, 1, index);
        end;

        index := Pos(',', sRecord);
        if index > 0 then
        begin
          SystemCoords[Row].fCoordY := StrToFloat(Copy(sRecord, 1, index - 1));
          Delete(sRecord, 1, index);
        end;

        SystemCoords[Row].fCoordZ := StrToFloat(sRecord);
      end;
    finally
      slList.Free;
    end;

  for i := Low(SystemCoords) to High(SystemCoords) do
  begin
    cbSystem.Items.Add(SystemCoords[i].szSystem);
  end;
end;

As you can see I am using "Pos" function to parse the csv file and also loop the array at the end to add the Star name to a combobox, Is there a more economical way of doing this?

Any suggestions are welcomed

Answer 1

Like others said, probably the majority of the time is spent populating the combo.

In my opinion, when dealing with big updates of a TStrings the BeginUpdate / EndUpdate technique proposed by the Jens Borrisholt's answer constitutes a valid approach.

---

As a minor issue, if your application is the only which writes and reads the data and neither machines nor humans care about the CSV format, you might consider to store the records adopting a different file format, using the BlockRead and BlockWrite functions.

type
  TStarCoords = record
    szSystem: string[40];
    fCoordX,
    fCoordY,
    fCoordZ: Single;
  end;

. . .

const
  CFILENAME = '<your path to some file .dat>';

Reading the data:

procedure TForm1.FormCreate(Sender: TObject);
var
  lstStarCoords: TList<TStarCoords>;
  f: File;
  starCoords: TStarCoords;
begin
  lstStarCoords := TList<TStarCoords>.Create;
  try

    AssignFile(f, CFILENAME);
    Reset(f, SizeOf(TStarCoords));
    try
      while not Eof(f) do begin
        BlockRead(f, starCoords, 1);
        lstStarCoords.Add(starCoords);
      end;
    finally
      CloseFile(f);
    end;

    cbSystem.Items.BeginUpdate;
    for starCoords in lstStarCoords do
      cbSystem.Items.Add(starCoords.szSystem);
    cbSystem.Items.EndUpdate;

  finally
    lstStarCoords.Free;
  end;
end;

Writing the data:

procedure TForm1.WriteStarCoords;
var
  lstStarCoords: TList<TStarCoords>;
  f: File;
  starCoords: TStarCoords;
  i: Integer;
begin
  lstStarCoords := TList<TStarCoords>.Create;
  try

    //let's insert 5k new items
    for i:=1 to 5000 do begin
      with starCoords do begin
        szSystem := 'HYEL YE';
        fCoordX := 122;
        fCoordY := 12.375;
        fCoordZ := 45.75;
      end;
      lstStarCoords.Add(starCoords);
    end;

    AssignFile(f, CFILENAME);
    Rewrite(f, SizeOf(TStarCoords));
    try
      for starCoords in lstStarCoords do
        BlockWrite(f, starCoords, 1);
    finally
      CloseFile(f);
    end;

  finally
    lstStarCoords.Free;
  end;
end;

---

EDIT: example using pointers to store the record information directly in the cbSystem component.

This approach is a little more "dangerous" since it allocates memory which has to be manually freed but allows to avoid the usage of a TDictionary to pair the TStarCoords.szSystem with the corresponding record.

Declare a new type which points to the TStarCoords record:

type
  PStarCoords = ^TStarCoords;

Reading the data:

procedure TForm1.FormCreate(Sender: TObject);
var
  lstStarCoords: TStringList;
  f: File;
  starCoords: PStarCoords;
begin
  ClearCbSystem;

  lstStarCoords := TStringList.Create(False);
  {another minor enhancement:
   since lstStarCoords does not own any TObject which needs to be freed
   the OwnsObjects property of the TStringList can be set to False
   in order to avoid some code to be execute in some method like Clear and Delete}
  try

    lstStarCoords.BeginUpdate;

    AssignFile(f, CFILENAME);
    Reset(f, SizeOf(TStarCoords));
    try
      while not Eof(f) do begin
        New(starCoords);
        BlockRead(f, starCoords^, 1);
        lstStarCoords.AddObject(starCoords^.szSystem, TObject(starCoords));
      end;
    finally
      CloseFile(f);
    end;

    lstStarCoords.EndUpdate;

    cbSystem.Items.Assign(lstStarCoords);
  finally
    lstStarCoords.Free;
  end;
end;

Clearing the list with cbSystem.Clear does not automatically dispose the underlying pointers which have to be manually freed. Use the ClearCbSystem procedure everytime the cbSystem list has to be cleared:

procedure TForm1.ClearCbSystem;
var
  i: Integer;
begin
  cbSystem.Items.BeginUpdate;
  for i := cbSystem.Items.Count-1 downto 0 do
    Dispose(PStarCoords(cbSystem.Items.Objects[i]));
  cbSystem.Clear;
  cbSystem.Items.EndUpdate;
end;

When the form is destroyed, a call to the ClearCbSystem procedure ensures the pointers are disposed before the cbSystem component is freed by the application itself:

procedure TForm1.FormDestroy(Sender: TObject);
begin
  ClearCbSystem;
end;