Fastest way to Convert Olevariant 2D Array of Double to Dynamic 2D Double Array

Question

I have a 2D OleVariant matrix of doubles xyInput := VarArrayCreate([0, Count-1, 0, 1], varDouble );.

I want a conversion (fast as possible) to a plain 2D dynamic array DestArray : array[0..1] of array of Double using move().

In this process of solving this, I have used Count=5 giving an expected 40 bytes per dimension. But I have discovered that the address diff between Pointer(DestArray[0]) and Pointer(DestArray[1]) is 56 bytes.

So what are the 16 bytes in between? I dont know about the first 8 bytes, but the last 8 bytes are information about the array dimension.

As a consequence the move() does not work in one step Move(VarArrayData^, Pointer(DestArray[0])^, BytesToMove );.

I have found a way by using to 2 separate moves, but I still have a feeling it can be done more elegant.

Questions:

1. Is there in fact a much more simple and easy fast way?
2. Are there any compiler directives or similar that can change the memory layout of the dynamic array to allow move() to work in a single step?
3. Out of curiosity, what are the first 8 bytes in the memory gap between Pointer(DestArray[0]) and Pointer(DestArray[1])?

Here is a complete code snippet:

procedure TForm1.FormCreate(Sender: TObject);

procedure PrintEqualityVerdictLine( value1 : Double; value2 : Double );
const
  cEqualVerdict : array[Boolean] of String = ( '!!!Not Equal!!!', 'Equal' );
begin
  Memo1.Lines.Add(FloatToStr(value1) + ' =? ' + FloatToStr(value2) + '      ' + cEqualVerdict[ SameValue( value1, value2, 0.001 ) ] );
end;

procedure VariantArrayOfDoubleToDynamicDoubleArray;
var
  xyInput : OleVariant;
  Count: Integer;
  n: Integer;

  DestArray : packed array[0..1] of packed array of Double;

  V_Ptr: PVarData;
  VarArrayData: PVarData;
  BytesToMove: Integer;

  SourceBytePtr : PByte;
  BytesToMovePerColumn: Integer;
  DestBytePtr: PByte;
begin
  // create 2 column OleVariant array:
  Count := 5;
  xyInput := VarArrayCreate([0, Count-1, 0, 1], varDouble );

  // fill test data:
  for n := 0 to Count-1 do
  begin
    xyInput[n, 0] := 1.0 * n;
    xyInput[n, 1] := 2.0 * Count + n;
  end;

  SetLength(DestArray[0], Count);
  SetLength(DestArray[1], Count);

  V_Ptr := PVarData(@xyInput);
  if ((V_Ptr^.VType and $F000 ) = varArray) and
     ((V_Ptr^.VType and varTypeMask ) = varDouble)
  then
  begin
    VarArrayData := PVarData(V_Ptr^.VArray^.Data);
    BytesToMovePerColumn := Count * V_Ptr^.VArray^.ElementSize;
    BytesToMove := BytesToMovePerColumn*V_Ptr^.VArray^.DimCount;

    // print 16 discovered intermediate bytes of the DestArray:
    DestBytePtr := Pointer(DestArray[0]);
    Inc(DestBytePtr, BytesToMovePerColumn);
    for n := 1 to 16 do
    begin
      Memo1.Lines.Add('byte['+IntToStr(n) + ']: ' + IntToStr( DestBytePtr^ ) );
      Inc(DestBytePtr);
    end;

//    This does NOT work: col 1 of arr gets offset due to 16 discovered intermediate bytes:
//        Move(VarArrayData^, Pointer(DestArray[0])^, BytesToMove );

//    This works:
    SourceBytePtr := PByte(VarArrayData);
    Move(SourceBytePtr^, Pointer(DestArray[0])^, BytesToMovePerColumn );
    Inc(SourceBytePtr, BytesToMovePerColumn);
    Move(SourceBytePtr^, Pointer(DestArray[1])^, BytesToMovePerColumn );
  end;

  // print:
  Memo1.Lines.Add('VariantArrayOfDoubleToDoubleArray:');

  Memo1.Lines.Add('col 0:');
  for n := 0 to Count - 1 do
    PrintEqualityVerdictLine( xyInput[n, 0], DestArray[0, n] );
  Memo1.Lines.Add('');

  Memo1.Lines.Add('col 1:');
  for n := 0 to Count - 1 do
    PrintEqualityVerdictLine( xyInput[n, 1], DestArray[1, n] );
end;

begin
  Memo1.Lines.Clear;
  VariantArrayOfDoubleToDynamicDoubleArray;
end;

Answer 1

But I have discovered that the address diff between Pointer(DestArray[0]) and Pointer(DestArray[1]) is 56 bytes.

That is very much to be expected. Well, to be more clear, there is no reason at all to expect that DestArray[0] and DestArray[1] will point to adjacent blocks of memory.

Your type is

array[0..1] of array of Double;

Note that I removed the packed keyword which is ignored when applied to arrays. What you have here is an array containing two pointers. These two pointers are independent. Look at how you allocate the dynamic arrays.

SetLength(DestArray[0], Count);
SetLength(DestArray[1], Count);

Each call to SetLength results in a separate heap allocation. No reason at all for the memory to be adjacent. That's before getting to the issue that a dynamic array has an extra block of meta data stored immediately before the payload of the array, and each block of memory has its own meta data used by the memory manager. So even if the memory manager by chance happened to serve up adjacent blocks of memory, the meta data would sit between the two arrays. Incidentally, this memory manager meta data is the answer to your question 3.

In technical terms, what you have here in DestArray is a jagged array. You on the other hand appear to be looking for a multi-dimensional array. Delphi does not actually support dynamic multi-dimensional arrays. All you have are jagged arrays. If you want a contiguous block of memory then you would need to allocate a one dimensional block of memory and perform the index calculation yourself.

So, as it stands, if you continue with jagged arrays then you will need to perform one copy for each inner array. If you switch to a linear array then you can get away with a single copy, but you will have to perform your own indexing. Of course, the indexing is very easy to do and that might be efficient. Finally, it's plausible that you could allocate a linear array in Delphi ahead of time, and put a pointer to that array into your variant and thereby avoid the copy completely.