Can I modify a constant in the RTL class System.Classes.TStream and rebuild it at runtime in Delphi XE6?

Question

I am trying to work around a known ugly performance limitation in System.Classes.pas, which has a 1980s era constant buffer limit ($F000) that looks like this:

function TStream.CopyFrom(const Source: TStream; Count: Int64): Int64;
const
  MaxBufSize = $F000;
....

This is causing major performance penalties in our Delphi application. In delphi XE2 through XE5, we were able to modify this and use one of the following approaches:

• I could modify the Delphi sources, and then, by invoking dcc32.exe from a batch file, rebuild the System.Classes.dcu file in the Delphi library folder. I realize this is ugly and I didn't like doing this, but I don't like this ugly performance issue in the RTL either, and our users can not live with the performance headaches it causes.

• I could try to put a modified system.classes.pas file somewhere in my project search path.

Neither of the above approaches is working for me in Delphi XE6, now, thanks probably to some internal compiler changes. The error I get in a minimal command line application that includes System.Contnrs in its uses clause, is this:

[dcc32 Fatal Error] System.Classes.pas(19600): F2051 Unit System.Contnrs was compiled with a different version of System.Classes.TComponent

The sample program to reproduce this problem (assuming you have modified System.Classes.pas and changed the MaxBufSize constant), is shown here:

program consoletestproject;

{$APPTYPE CONSOLE}

{$R *.res}

uses
   System.Contnrs,
   System.SysUtils;

var
  List:System.Contnrs.TObjectList;
begin
   WriteLn('Hello world');
end.

Again, this problem reproduces easily in Delphi XE6, but is not a problem in XE5, or earlier.

What is the recommended practice when you absolutely MUST work around a fundamental RTL or VCL limitation using a modified copy of System.Classes.pas or System.SysUtils.pas or some other very low level unit? (Yes, I know you should NOT do this if you don't have to, don't bother with a lecture.)

Are there a magic set of command line parameters you can use via "dcc32.exe" on the command line, to produce a modified DCU that will link properly with the application example above?

As a secondary question, are there .dcu files for which no source exists that will break when one tries to do this, in which case the answer to all of the above is, "you can't fix this, and if there's a bug in the RTL, you're out of luck"?

One possible workaround is to include "$(BDS)\source\rtl\common" in your project search path (or library path), forcing each broken (needing recompile) DCU to rebuild EACH time, but this seems ugly and wrong.

Answer 1

You can overcome this limitation using a detour, try this sample which uses the Delphi Detours Library

First define the signature of the method to hook

var
 Trampoline_TStreamCopyFrom : function (Self : TStream;const Source: TStream; Count: Int64): Int64 = nil;

then implement the detour

function Detour_TStreamCopyFrom(Self : TStream;const Source: TStream; Count: Int64): Int64;
const
  MaxBufSize = 1024*1024; //use 1 mb now :)
var
  BufSize, N: Integer;
  Buffer: TBytes;
begin
  if Count <= 0 then
  begin
    Source.Position := 0;
    Count := Source.Size;
  end;
  Result := Count;
  if Count > MaxBufSize then BufSize := MaxBufSize else BufSize := Count;
  SetLength(Buffer, BufSize);
  try
    while Count <> 0 do
    begin
      if Count > BufSize then N := BufSize else N := Count;
      Source.ReadBuffer(Buffer, N);
      Self.WriteBuffer(Buffer, N);
      Dec(Count, N);
    end;
  finally
    SetLength(Buffer, 0);
  end;
end;

Finally replace the original function by the trampoline (you can use this code in the initialization part of some unit)

  Trampoline_TStreamCopyFrom     := InterceptCreate(@TStream.CopyFrom,   @Detour_TStreamCopyFrom);

And to release the hook you can use

 if Assigned(Trampoline_TStreamCopyFrom) then
   InterceptRemove(@Trampoline_TStreamCopyFrom);