How to marshal this c struct and call its method in c#

Question

I've seen answers similar to this in other places, but none of those solutions are working and usually they are slightly different questions. I haven't seen any of the other questions' solutions help here.

I have a dll with a method-parameter of a type of struct containing a uint array, passed to the method as a pointer. Here's what the old c header has:

typedef struct HandleArrayType
{
    uint32_t HandleArray[max_number_100];
}HandleArrayType;
typedef HandleArrayType *HandleArrayPtrType;

typedef bool (*type_connectToMultiple)(char *CommaSeparatedString, HandleArrayPtrType ArrayPointer, uint32_t numberOfOpensAttempted) __attribute__((cdecl));

We've tried several ways to marshal this, here's the last suggestion we tried and was giving us memory access violations:

[StructLayout(LayoutKind.Sequential)]
public struct HandleArrayType
{
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 100)]
    public uint[] HandleArray;
}

[DllImport("file", CallingConvention = CallingConvention.Cdecl)]
internal static extern bool ConnectToMultiple(string[] nameArray, ref HandleArrayType myStruct, uint numAttempted);

I think I also tried something like this before a few times, which does connect, but we were getting one handle instead of two and the handle number was way off (indicating improper marshaling of the should-be struct):

[DllImport("file", CallingConvention = CallingConvention.Cdecl)]
    internal static extern bool ConnectToMultiple(IntPtr commaSepHandlesString, uint[] handles , ref uint numAttempted);

(Then when we call this we marshal as an IntPtr for the handles string) If anyone knows of a way to marshal this properly please post a solution, we are using a crappy workaround right now that will make our process very slow (and speed is a concern).

EDIT: We were able to make our workaround faster by installing a more recent version of the dll, so it isn't a huge problem for us anymore. But the connectToMultiple method is still busted. Still curious to see if it can be made to work

Answer 1

It seems the C Declaration is wrong, and numAttempted is a pointer. Furthermore, you need to specify the charset of the string, which is normally LPStr.

Furthermore, it's unclear if myStruct should be [In, Out] ref or [Out] out or in.

[StructLayout(LayoutKind.Sequential)]
public struct HandleArrayType
{
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 100)]
    public int[] HandleArray;
}

[DllImport("file", CallingConvention = CallingConvention.Cdecl)]
internal static extern bool ConnectToMultiple(
    [MarshalAs(UnmanagedType.LPStr)] string nameArray,
    [In, Out] ref HandleArrayType myStruct,
    [Out] out uint numAttempted);

static void Main(string[] args)
{
    var handleArrayType = new HandleArrayType();
    bool results = ConnectToMultiple("abc", ref handleArrayType, out var numAttempted);
}