Wrap native DLL for C#

Question

I wrote a C++ DLL and now I need to call a native function from a managed app.

The exported native function appears like this:

extern "C" __declspec(dllexport) 
bool NativeMethod(char *param1, char *param2, char *result);

So, from C# I'll call that function passing 2 input params, 1 output param and obviously I'll read the return bool value.

I tried to wrap all this in many ways, but always I get a PInvokeStackImbalance exception. The only way I know to call native function is by applying CallingConvention = CallingConvention.Cdecl) on .NET function declaration. However in this way I'm not able to read the output param (it's empty string always) and also the return value is always true.

Answer 1

First, I'd adjust the prototype of your native function.

Since this function has a C interface, you should use a C type for booleans, not a C++ type like bool. You may want to use Win32's BOOL type.

Moreover, as it currently is, your function is prone to buffer overruns: it's better to add another parameter to specify the maximum size of the destination result string buffer.

Note also that a widespread calling convention for DLLs exporting pure C interface functions (like lots of Win32 API functions) is __stdcall (not __cdecl). I'd use that as well.

Last, since the first two parameters are input strings, you may want to use const to make it clear and enforce const-correctness.

So, I'd make the prototype of the exported native function like this:

extern "C" __declspec(dllexport) 
BOOL __stdcall NativeFunction(
    const char *in1, 
    const char *in2, 
    char *result, 
    int resultMaxSize);

Then, on the C# side, you can use the following P/Invoke:

   [DllImport(
        "NativeDll.dll", 
        CharSet = CharSet.Ansi, 
        CallingConvention = CallingConvention.StdCall)]
    [return: MarshalAs(UnmanagedType.Bool)]
    static extern bool NativeFunction(
        string in1,
        string in2,
        StringBuilder result, 
        int resultMaxSize);

Note that for the output string, a StringBuilder is used.

Note also that CharSet = CharSet.Ansi is used to marshal C#'s Unicode UTF-16 strings to ANSI (pay attention to the fact that the conversion is lossy - if you want a non-lossy conversion, just use wchar_t* strings on the C++ side as well).

I did a test with a simple C++ native DLL:

// NativeDll.cpp

#include <string.h>
#include <windows.h>

extern "C" __declspec(dllexport) 
BOOL __stdcall NativeFunction(
    const char *in1, 
    const char *in2, 
    char *result, 
    int resultMaxSize)
{
    // Parameter check
    if (in1 == nullptr 
        || in2 == nullptr 
        || result == nullptr 
        || resultMaxSize <= 0)
        return FALSE;

    // result = in1 + in2
    strcpy_s(result, resultMaxSize, in1);
    strcat_s(result, resultMaxSize, in2);

    // All right
    return TRUE;
}

And it is called successfully by the following C# console app code:

using System;
using System.Runtime.InteropServices;
using System.Text;

namespace CSharpClient
{
    class Program
    {
        [DllImport(
            "NativeDll.dll", 
            CharSet = CharSet.Ansi, 
            CallingConvention = CallingConvention.StdCall)]
        [return: MarshalAs(UnmanagedType.Bool)]
        static extern bool NativeFunction(
            string in1,
            string in2,
            StringBuilder result, 
            int resultMaxSize);

        static void Main(string[] args)
        {
            var result = new StringBuilder(200);
            if (! NativeFunction("Hello", " world!", result, result.Capacity))
            {
                Console.WriteLine("Error.");
                return;
            }

            Console.WriteLine(result.ToString());
        }
    }
}