Delphi scanning for database directories using multi threading

Question

I need to scan drives for directories containing my database files to add them to the BDE paths. How can I make my code do it faster using threads? I'm using delphi 2007, so omniThread is not supported. I need to know how to make the thread, and how to execute it. This is my Code: thanks.

procedure TMainFrm.RestoreDBDirs;
var
  Lst: TStringList;
  Dirs: string;
  Counter, j, LstFrom, LstTo: integer;
  SearchRec: TSearchRec;
  ST: TScanThread;    
begin
  Screen.Cursor:= crHourGlass;
  try
    try
      ChangeAlias(AliasCombo);//After this procedure the tables are closed
    except
    end;
    Lst:= TStringList.Create;
    Lst.Clear;
    Counter:= 0;

if Assigned(ChooseDrvFrm) then
with ChooseDrvFrm do
begin
  Lst.Add(lvDrives.Selected.Caption);
  Dirs:= lvDrives.Selected.Caption;

  Progress1.Position:= 0;
  Progress1.Visible:= True;
  stBar1.SimpleText:= 'Searching for Databases...';
end
else
begin
  Lst.Add(GetSystemDrive);
  Dirs:= GetSystemDrive;
end;

repeat
  // Update Progress Bar
  if Assigned(ChooseDrvFrm) then
  with ChooseDrvFrm do
  begin
    Progress1.StepBy(1);
    if Progress1.Position = Progress1.Max then
      Progress1.Position:= 0;
  end;

  Dirs:= Lst.Strings[Counter] +'\';
  if (Dirs <> '.') and (Dirs <> '..')then
    if FindFirst(Dirs +'*.*', faDirectory, SearchRec) = 0 then
    begin
      if ((SearchRec.Attr and faDirectory) > 0)
      and (SearchRec.Name <> '.') and (SearchRec.Name <> '..') then
      begin
        Lst.Add(Dirs + SearchRec.Name);

        if Assigned(ChooseDrvFrm) then
          ChooseDrvFrm.stBar1.SimpleText:= Dirs + SearchRec.Name;
      end;

      while FindNext(SearchRec) = 0 do
        if ((SearchRec.Attr and faDirectory) > 0) and
            (SearchRec.Name <> '.') and (SearchRec.Name <> '..') then
        begin
          Lst.Add(Dirs + SearchRec.Name);
          if Assigned(ChooseDrvFrm) then
            ChooseDrvFrm.stBar1.SimpleText:= Dirs + SearchRec.Name;
        end;
    end;
  Counter:= Counter + 1;
  FindClose(SearchRec);
until Counter = Lst.Count;

Dirs:= '';

if Assigned(ChooseDrvFrm) then
  ChooseDrvFrm.Progress1.Position:= 0;

for Counter:= 0 to Lst.Count - 1 do
begin
  if Assigned(ChooseDrvFrm) then
  with ChooseDrvFrm do
  begin
    Progress1.StepBy(1);
    if Progress1.Position = Progress1.Max then
      Progress1.Position:= 0;
  end;

  if (FileExists(Lst.Strings[Counter] + '\Crt.DB'))
  and (FileExists(Lst.Strings[Counter] + '\Ds.DB'))
  and (FileExists(Lst.Strings[Counter] + '\Turim.DB'))
  and (FileExists(Lst.Strings[Counter] + '\Rprt.DB'))
  and (UpperCase(Lst.Strings[Counter]) <> UpperCase('C:\My Installations\Data'))
  and (UpperCase(Lst.Strings[Counter]) <> UpperCase(ExtractFileDir(ParamStr(0)))) then
  try
    if Assigned(ChooseDrvFrm) then
      ChooseDrvFrm.stBar1.SimpleText:= 'Restoring Databases: '+ Lst.Strings[Counter];

    RestoreAlias(Lst.Strings[Counter]);
  except
    on EDatabaseError do;
  end;
end;

if Assigned(ChooseDrvFrm) then
with ChooseDrvFrm do
begin
  Progress1.Position:= 0;
  Progress1.Visible:= False;
  stBar1.SimpleText:= 'Done';
  MessageDlg('Databases succesfully restored', mtInformation, [mbYes], 0);
  Close;
end;

>     FillAliasCombo;   finally
>     Lst.Free;
>     Screen.Cursor:= crDefault;   end;

Answer 1

For traditional spinning disks threading won't help you. Your task is disk bound rather than CPU bound, and threading will lead to inefficient disk head movements. Attempting to do this with multiple threads will likely be slowed than a single thread.

For solid state drives, or network drives then your process is still disk bound. However, using threads to perform tasks in parallel can alleviate some of the latencies of the disk access process and yield improved performance.

It will take some experimentation to work out how best to scan the disks and will likely require different strategies for different disk types.

I think a producer/consumer approach would be the first thing to try. A producer thread that enumerates the directories. And then multiple consumers to read the contents of those directories. You would likely want to batch up multiple directories into a single task to minimize the impact of the threading overhead.