Handeling NULL pointer out argument in C# COM

Question

I am working with a COM plugin interface that has the following function definition:

HRESULT foo ( [out, ref] VARIANT* a, [out, ref] VARIANT* b );

When using tlbimp (tlbimp2.exe from codeplex) the .NET library has the following interface function:

int foo ( out object a, out object b );

The problem is that the calling application will make the function call:

VARIANT a;
::InitVariant( &a );

plugin->foo( &a, NULL );

And in C# I implemented the function:

int foo ( out object a, out object b )
{
    a = 1;
    b = 2;

    return 0; // S_OK
}

When all is said and done the application actually gets a E_POINTER return and not S_OK. I assume it is because of the NULL passed the the out parameter.

Is there a way to check if the address pointer is NULL in the C# implementation?

Note: out parameters are not initialized so you cant use the parameters at all.

I've tried implementing the interface as [in, out, ref] to force C# to use ( ref object a, ref object b ) but that did not work either.

Update

Hans is absolutely correct that we should have been calling the function with a NULL ptr if we were declaring it as [out,ref].

erurainon was also correct that we could just use IntPtr to get to the Variant*.

So here is how it was fixed:

int foo ([MarshalAs(UnmanagedType.Struct)]out object a, [MarshalAs(UnmanagedType.Struct)]out object b);

became

int foo ([MarshalAs(UnmanagedType.Struct)]out object a, IntPtr b );

Now we can test for the NULL cases with:

if ( b == IntPtr.Zero )

However since we are dealing with Variant you can't just copy the value to the IntPtr like:

Marshal.StructureToPtr( myValue, b, false );

So following this post, you need to make a struct class:

[StructLayout(LayoutKind.Explicit, Size = 16)]
public struct PropVariant
{
    [FieldOffset(0)]
    public VarEnum variantType;
    [FieldOffset(8)]
    public IntPtr pointerValue;
    [FieldOffset(8)]
    public byte byteValue;
    [FieldOffset(8)]
    public long longValue;
    [FieldOffset(8)]
    public double dateValue;
    [FieldOffset(8)]
    public short boolValue;
}

And the final function looks like this:

int foo( out object a, IntPtr b )
{
    a = 100;

    if ( b != IntPtr.Zero )
    {
        var time = new PropVariant();
        time.dateTime = DateTime.Now.ToOADate();
        time.variantType = VarEnum.VT_DATE;

        Marshal.StructureToPtr( time, b, false );
    }

    return 0; // S_OK
}

Hope this helps someone else in the future

Answer 1

Hans is right. Either you disallow your API to be called with NULL parameters, or you implement your C# function with IntPtr arguments and check for IntPtr.Zero.