Search code examples
.netvb.netpinvokemarshalling

VB.NET Pinvoke: How to fix the structure address passed into DLL?

I have a DLL written in C (compiled using VC++2017). There are several functions accept pointers to a structure.

During the init call, it will save the address passed in. And during later calls, the DLL expects that the passed in addresses is the same as the first init call.

In vb.net, I define a structure (packed as 4), I checked the memory layout, it's exactly the same as C when passed into the DLL.

However, each time I called a function using the structure (ByRef), the address may or may not change (shift for 4 bytes).

Am I missing anything Or is it even possible to do that in VB.NET?

The codes are as follows, the c struct (It's legacy code and I'd prefer not to change this),

struct A
{
    char a[9] ;
    char b[9] ;
    char c[2] ;
    char d[9] ;
    int e;
    int f;
    char g[2] ;
    char h[9] ;
    int i;
    int j;
    char k[2] ;
    int l;
    char m[41] ;
    char n[41] ;
    char o[10] ;
} ;

This is what I defined in VB.NET,

<StructLayout(LayoutKind.Sequential, Pack:=4)>
Structure A
    <VBFixedArray(9), System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValArray, SizeConst:=9)> Dim a() As Byte
    <VBFixedArray(9), System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValArray, SizeConst:=9)> Dim b() As Byte
    <VBFixedArray(2), System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValArray, SizeConst:=2)> Dim c() As Byte
    <VBFixedArray(9), System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValArray, SizeConst:=9)> Dim d() As Byte
    Dim e As Integer
    Dim f As Integer
    <VBFixedArray(2), System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValArray, SizeConst:=2)> Dim g() As Byte
    <VBFixedArray(9), System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValArray, SizeConst:=9)> Dim h() As Byte
    Dim i As Integer
    Dim j As Integer
    <VBFixedArray(2), System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValArray, SizeConst:=2)> Dim k() As Byte
    Dim l As Integer
    <VBFixedArray(41), System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValArray, SizeConst:=41)> Dim m() As Byte
    <VBFixedArray(41), System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValArray, SizeConst:=41)> Dim n() As Byte
    <VBFixedArray(10), System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValArray, SizeConst:=10)> Public o() As Byte
End Structure

These are the C prototype:

__declspec( dllexport ) int __stdcall init(struct A * param1)
__declspec( dllexport ) int __stdcall dosomething(struct A * param1)

These are the VB.NET prototype

Public Declare Function init Lib "A.dll" (ByRef param1 As A) As Integer
Public Declare Function dosomething Lib "A.dll" (ByRef param1 As A) As Integer

Dim a As New A
'ok
init(a)
' ok
dosomething(a)
' The second call to dosomething, the param1's address changed by 4 bytes
dosomething(a)

The above is just the simplified version. You get the idea that the param1's address in C changes during the different calls.

Is there a way to solve this?

Thanks.

Solution

  • It's perfectly natural that the address of the marshaled struct is different in each call. That's because the marshaler has to create an unmanaged struct to send to the unmanaged code. The managed structure does not have the same layout as the unmanaged struct which makes this necessary. And even if the managed and unmanaged structs had compatible layout (i.e. the struct was blittable), the address might change because the .net memory manager can move objects.

    You can however, take charge of the marshaling process. Allocate some unmanaged memory (e.g. by calling Marshal.AllocHGlobal, and then use Marshal.StructureToPtr to populate that memory with a marshaled version of the structure. You can then pass the address of that unmanaged memory to the unmanged code. When you have finished all your calls to the unmanaged code, call Marshal.PtrToStructure to read any modifications made to the structure.

    Perhaps the bigger question is why feel the need for the address to be stable between calls. I'm finding it very hard to imagine a scenario where that is a reasonable expectation to put on the caller. Is it possible that your unmanaged code is taking liberties by requiring this of the caller?