Multithreaded WinHttp downloads

Question

I am creating a Delphi application to download files from the Internet and if the server supports range requesting it will be multi threaded. The progress is also relayed back to the GUI.

The current software model uses TThread components. The GUI calls a TDownloadThread which then spawns the TDownloadPartThreads - these are the threads which actually do the downloading over 'WinHttp'.

My problem: The CPU is used up, even for one download where there are only 4 threads downloading.

My Supposed Causes:

1. I am writing to the destination file every 8192 bytes, and was wondering if I should buffer it before writing in one block?
2. The thread communication is done via Synchronize(MainForm.UpdateProgress(Downloaded, TotalSize)) which I have heard is AWFUL to do, maybe I should share an object between the threads so I can access this using a timer on the main form, to update progress?

My Solutions

1. Stagger the file writing and only write every x bytes.

2. Update the TThread components to use OmniThreadLibrary and send the data back to the main form somehow. Each TDownloadPart thread would then become an IOmniWorker and send back its progress by sharing an object with the main form. The main form would then use a timer to update its progress, like: ProgressBar1.Position := sharedDataObject.Progress;

Hopefully someone can point me in the right direction!

Answer 1

I would use shared object to update state - just like you suggest in your second solution. If you only share an 8-byte (4 is not enough!) file size and if you make sure that address of each shared location is 8-aligned you can use interlocked instructions to modify this shared state and you won't event need locking.

The simplest way to maintain shared state would be TGp8AlignedInt64 record from the GpStuff unit, which would work equally well with OmniThreadLibrary- or TThread-based solution.

TGp8AlignedInt64 = record
  function  Add(value: int64): int64; inline;
  function  Addr: PInt64; inline;
  function  CAS(oldValue, newValue: int64): boolean;
  function  Decrement: int64; overload; inline;
  function  Decrement(value: int64): int64; overload; inline;
  function  Increment: int64; overload; inline;
  function  Increment(value: int64): int64; overload; inline;
  function  Subtract(value: int64): int64; inline;
  property Value: int64 read GetValue write SetValue;
end; 

All operations on this record are thread-safe, so you can safely do .Add in the worker thread and call .Value from the timer event in the main form at the same time.