How to pass an array from C# to unmanaged COM by reference (VT_BYREF)

Question

I have a COM component written in C++ whose source I can't change, and one of the arguments to one of its methods is VARIANT *pParamArray. Using tlbimp I can create a managed stub for it and pass it an array from C#.

Unfortunately, the COM component is expecting its array to be passed by reference - there's an explicit check for pParamArray->vt != (VT_BYREF | VT_ARRAY | VT_VARIANT) and it returns an error if it doesn't pass that check.

I have the PDB and source for the COM component, so I'm debugging both the C# and the unmanaged code in tandem. I can see that my C# array of object[] is being passed as VT_ARRAY | VT_VARIANT, which is essentially a SAFEARRAY as far as I understand it.

How can I explicitly tell C# that I want to pass it by reference, so that the type on the far end has the VT_BYREF mask?

• I've tried putting it in a VariantWrapper - I get an ArgumentException with the message "VariantWrappers cannot be stored in Variants."
• I've tried doing a Marshal.AllocHGlobal and using Marshal.GetNativeVariantForObject() but I only get an int on the COM end.

tlbimp by default marshals the parameter in question as UnmanagedType.Struct. I'm not sure how to make tlbimp marshal it as IntPtr, or even if this would make a difference (I also tried using the enhanced tlbimp2 from CodePlex, but it doesn't seem to recognize my request for IntPtr in its config file).

I'm by no means an Interop expert so feel free to suggest something which may appear obvious to you.

Update 1

As requested by @ZdeslavVojkovic, here are relevant pieces of the IDL:

[
    uuid(01234567-89AB-CDEF-0123-3456789ABCDE),
    version(1.0),
    helpstring("XXX")
]
library LAbc
{
    [
        object,
        uuid(01234567-89AB-CDEF-0123-3456789ABCDE),
        dual,
        helpstring("XXX"),
        pointer_default(unique)
    ]
    interface IAbc : IDispatch
    {
            [id(1), helpstring("XXX")]
            HRESULT CallFunction([in] myEnum Function, [in, out] VARIANT* pParamArray, [out, retval] long* pVal);
    };

    [
        uuid(01234567-89AB-CDEF-0123-3456789ABCDE),
        helpstring("XXXs")
    ]
    coclass Abc
    {
        [default] interface IAbc;
    };
};

And here is the method signature itself and the internal check for the parameter's type:

STDMETHODIMP XAbc::CallFunction(myEnum Function, VARIANT *pParamArray, long *pVal)
{
    ...

    // we must get a pointer to an array of variants
    if(!pParamArray ||
        (pParamArray->vt != (VT_BYREF | VT_ARRAY | VT_VARIANT)) ||
        !(psa = *pParamArray->pparray))
        return E_INVALIDARG;

    ...
}

Answer 1

Here's how to make it work without rewriting the IL.

Please note that for simplicity I have skipped the enum param so IDL definition of the method is like this:

[
    object,
    uuid(E2375DCC-8B5B-4BD3-9F6A-A9C1F8BD8300),
    dual,
    helpstring("IDummy Interface"),
    pointer_default(unique)
]
interface IDummy : IDispatch
{
    [id(1)] HRESULT Fn([in, out] VARIANT *pParamArray, [out, retval]long *pVal);
};

You can call it by late binding call like this:

INTEROPXLib.IDummy d = new INTEROPXLib.DummyClass();

object data = new object[3]; // method argument, i.e. pParamArray value

var t = typeof(INTEROPXLib.IDummy);
object[] args = new object[1]; // array which will contain all method arguments
args[0] = data; // data is the first argument, i.e. first element of args array

ParameterModifier[] pms = new ParameterModifier[1];
ParameterModifier pm = new ParameterModifier(1);
pm[0] = true; // pass the 1st argument by reference
pms[0] = pm;  // add pm to the array of modifiers 

// invoke Fn by name via IDispatch interface
var ret = t.InvokeMember("Fn", System.Reflection.BindingFlags.InvokeMethod, null, d, args, pms, null, null);
Console.Out.WriteLine("Result = " + ret);

For convenience, it would be better to wrap this into an extension method on the interface.