Crash "at <unknown> <0xffffffff>" when calling C# method from C which returns a value

Question

I'm writing a native library in Go and C (thin wrapper) and interop code in C#.

I can successfully register C# methods as callbacks with the library and then call them from C when they don't return values. But I get a crash whenever I try to call callbacks which return values.

The library is compiled as a Windows DLL and here is the stack trace produced when I attempt to call a C# function in a Unity game:

=================================================================
    Native Crash Reporting
=================================================================
Got a UNKNOWN while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

=================================================================
    Managed Stacktrace:
=================================================================
      at <unknown> <0xffffffff>
      at MyLib.Interop:TestCallGetIntCb <0x000cd>
      at MyLib.Test:TestCallGetIntCb <0x0007a>
      at Multiplayer:Update <0x000ea>
      at System.Object:runtime_invoke_void__this__ <0x00187>
=================================================================

Commenting out the callback call in the C code removes the error.

Here is the code which produces the crash:

Native library:

callback.h

typedef int (*GetIntCallback)();

void SubscribeGetIntCallback(GetIntCallback cb);
int GetInt();

callback.c

static GetIntCallback testGetIntCallback = 0;

void SubscribeGetIntCallback(GetIntCallback cb)
{
    testGetIntCallback = cb;
}

int GetInt()
{
    if (testGetIntCallback != 0)
    {
        return testGetIntCallback();
    }

    return 0;
}

C# Code:

Interop.cs

public class Interop
{
    // Other code removed for clarity...

    public delegate int GetIntCallback();

    [DllImport("mylibrary")]
    public static extern void SubscribeGetIntCallback(GetIntCallback intCb);
}

Test.cs

public class Test
{
    // Other code removed for clarity...

    public Test()
    {
        Interop.SubscribeGetIntCallback(TestIntCb);
    }

    [MonoPInvokeCallback(typeof(Interop.GetIntCallback))]
    private int TestIntCb()
    {
        return 100;
    }
}

For reference the code below works and does not crash:

Native library:

callback.h

typedef void (*StringCallback)(const char *message, int size);

void SubscribeLogInfo(StringCallback cb);
void LogInfo(const char *message);

callback.c

static StringCallback logInfoCallback = 0;

void SubscribeLogInfo(StringCallback cb)
{
    logInfoCallback = cb;
}

void LogInfo(const char *message)
{
    if (logInfoCallback != 0)
    {
        logInfoCallback(message, (int)strlen(message));
    }
}

C# code:

Interop.cs

public class Interop
{
    // Other code removed for clarity...

    public delegate void StringCallback(IntPtr ptr, int size);

    [DllImport("mylibrary")]
    public static extern void SubscribeLogInfo(StringCallback cb);
}

Events.cs

public static class Events
{
    // Other code removed for clarity...

    public delegate void MessageEventHandler(object sender, MessageEventArgs args);
    public static event MessageEventHandler LogInfoEvent;

    static Events()
    {
        Interop.SubscribeLogInfo(OnLogInfo);
    }

    [MonoPInvokeCallback(typeof(Interop.StringCallback))]
    private static void OnLogInfo(IntPtr messagePtr, int size)
    {
        string message = Marshal.PtrToStringAnsi(messagePtr, size);
        LogInfoEvent?.Invoke(typeof(Events), new MessageEventArgs(message));
    }
}

Why do I get the crash when calling the method which returns an int, but not when I call the method which returns void?

My ideas:

• Is there a problem with my method signature?
• Is it because my Test class is not static?
• Is it because my TestIntCb() method is not static?
• Is it because the return value of TestIntCb() is in managed memory and not copied to unmanaged memory when it returns?
• Is there a type compatibility problem? int's are blittable aren't they, shouldn't it be copied with no conversion?

Thank you for your time!

Answer 1

You have two issues I can see here:

• Missing calling convention both on the DllImport and on the delegat declaration, which I guess should be CDecl.
• You need to ensure the delegate object doesn't get disposed/collected too early. In most cases the easiest way to do this is by caching it in a field. Note that the field itself (or rather its containing object) must also not be collected, but you don't show how you are using that.

public class Interop
{
    [UnmanagedFunctionPointer(CallingConvention.Cdecl)]
    public delegate int GetIntCallback();

    [DllImport("mylibrary", CallingConvention = CallingConvention.CDecl)]
    public static extern void SubscribeGetIntCallback(GetIntCallback intCb);
}

public class Test
{
    GetIntCallback _callback = TestIntCb;

    public Test()
    {
        Interop.SubscribeGetIntCallback(_callback);
    }

    [MonoPInvokeCallback(typeof(Interop.GetIntCallback))]
    private int TestIntCb()
    {
        return 100;
    }
}

---

On the same basis, the string version could also do with changes.

I note that you have put the event subscriber in the static constructor. This is probably a bad idea, put it into a separate function instead. Static events are also a bit of a bad idea, they need careful management to ensure you don't get a memory leak.

public class Interop
{
    [UnmanagedFunctionPointer(CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
    public delegate void StringCallback(string str, int size);

    [DllImport("mylibrary", CallingConvention = CallingConvention.Cdecl)]
    public static extern void SubscribeLogInfo(StringCallback cb);
}

public static class Events
{
    public delegate void MessageEventHandler(object sender, MessageEventArgs args);

    public static event MessageEventHandler LogInfoEvent;

    private StringCallback _callback = OnLogInfo;

    static Events()
    {
        Interop.SubscribeLogInfo(_callback);
    }

    [MonoPInvokeCallback(typeof(Interop.StringCallback))]
    private static void OnLogInfo(string message, int size)
    {
        LogInfoEvent?.Invoke(typeof(Events), new MessageEventArgs(message));
    }
}