C# calling deviceIOControl with complex structures

Question

So I am trying to write a C# wrapper to talk to one of our device drivers. (creating unit test) The driver is new, but coded against old c++ headers, so the structure layouts are defined, and can't really change.

So I have replicated the c++ structures the device is expecting DeviceIOControl to pass in.

Update #3 - changing the code to demo code that has the same issue. Also cleaning up the question to be more usable to others, see my answer below

[StructLayout(LayoutKind.Sequential, Pack=1)]
public class Points
{
    public int id;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst=10)]  
    public int[] x = new int[10];
    [MarshalAs(UnmanagedType.ByValArray, SizeConst=10)]
    public int[] y = new int[10];
};

[StructLayout(LayoutKind.Sequential, Pack=1)]
public class Shape
{
    public int name;
    [MarshalAs(UnmanagedType.ByValArray, SizeConst=10)]
    public Points[] p = new Points[10];
};

[StructLayout(LayoutKind.Sequential,Pack1)]
public class GeoShape:Shape
{
    public int top;
    public int left;
    [MarshalAs(UnmanagedType.ByValArray, SizeConst=10)]
    public int[] v = new int[10];
};

My call to deviceIOControl fails, because on the driver side it checks the size of the buffer passed in. On the C# side, I my object is too small as Marshal.SizeOf() returns 52 as a size, when it should be 852, if I add the Size= to the StructLayout attribute, the function will "pass", but I am fairly sure the data is not being correctly passed.

I am fairly sure the issue is this public Points[] p = new Points[10]; I think Marshal.StructToPtr() is not correctly marshaling this as it essentially a multi-dimensional array.

So I guess my questions is this even possible? Seems like C# could be smart enough to know how to create the right amount of space in memory for that array of structure.. but maybe not?

Alternatives I thought of that "could" work.

1. Write custom serailizers that convert the object to a byte[] and back, with zero meta-data. - Not ideal.

2. Would it be possible to write a mixed clr c++ dll and try to use that as an wedge. However my concerns, am I just going to have the same issue, but just in managed c++? Or even in mixed mode I would have to write a managed class to wrap the un-manged object to use it in c#. But the issue becomes how to pass that into deviceIOcontrol, if I do it from c# than will have the current issue of trying to marshal stuff correctly? Or if I pass it into a C++ call that calls DeviceIOControl, than i need to know how to get to the un-manged type of each manged object passed in.

3. Just write c++ functions that create the objects and call deviceIOControl, less idea as the parameters could get out of control?

4. Give up and do it all in C++, I am actually try to write unit test for my hardware, and newer cpp unit test in VS do integrate fairly nicely...

I also saw this earlier question, and gave it a try, however I think my scenerio is a little different. Un-/Marshalling nested structures containing arrays of structures

 struct Points
{
    int id;
    int x[10];
    int y[10];
};


struct Shape
{
    int name;
    Points p[10];
};

struct GeoShape :Shape
{
    int top;
    int left;
    int v[10];
};

Updated 2 I should clarify that I am trying to send an object to the driver, not receive one back (not yet atleast)

this is how I am calling it.

public static bool SetObject(SafeFileHandle device, DeviceControlCode ioctlCode, Object obj)
        {
            int size = Marshal.SizeOf(obj.GetType());
            IntPtr ptr = Marshal.AllocHGlobal(size);
            Marshal.StructureToPtr(obj, ptr, false);

            // call the dviceIOControl method
            return Control(device, ref ioctlCode, ptr, size, IntPtr.Zero, 0);
        }

Answer 1

I don't really know what's on the headers but as a starting point consider reading this about when to use struct or class

As an alert... are you sure about Pack = 1? Do you have a #pragma setting it to 1?

If you provide the related .h code it would be easier to check what might be wrong. Anyway, with the available information, I would do it like this:

[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct VENUS_FORMAT4
{
    public uint Top;    
    public uint Left;                                                           
    public uint Rows;                                                           
    public uint Columns;                                                        
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = Constants.MAX_CD_ROWS)]
    public uint[] V65Rows;                    
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = Constants.MAX_CD_COLS_DD2)]
    public uint[] CDCols;                 
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = Constants.MAX_DD_SECTIONS)]
    public uint[] DDSections;             
}

The rest is basically the same as above except in VENUS_VM4_DEVICE_FORMAT4IL which you will have to "replicate" the fields because you cannot inherit when using structs (in C# (type values)).

Also, if on the C++ side you have unions this is not going to work and you should use LayoutKind.Explicit and FieldOffset.