Passing String from C# to C++ DLL different Text Encoding on different return Types

Question

I hope somebody can explain what exactly the difference is:

In my C# Programm I want to pass an String to an C++ Method. My Code in C# looks like this:

    [DllImport("Test.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.StdCall)]
    public static extern String DoIt(String filename);

The inside of my C++ DLL looks like this:

__declspec(dllexport) CString DoIt(char* szFilename)
{
....
}

If I pass the String (like: C:\temp\my.txt) it becomes malformed =>"Ôœ&°8é-".

Now comes the confusings part I can't literly understand. If I change the return Type from CString to char* everything is fine.

__declspec( dllexport ) char* DoIt(char* filename)

Why is that so? The CharSet in C# is already set to Ansi to Marshal the String into the right Type. I cannot figure out where the connection between the return Type and my passing String is.

If you need more information just let me know.

Answer 1

Both versions of your code are wrong. You certainly can't pass CString as an interop type. You to use simple types for interop, not C++ classes.

The error when you use char* is more subtle. In that scenario there are two problems.

Firstly, with a string return type on the C# side, the pinvoke marshaller assumes that the returned string was allocated with CoTaskMemAlloc and will call CoTaskMemFree to deallocate it once it has been copied.

Secondly, although we can't see it, your C++ code almost certainly returns a pointer to a buffer owned by a local variable in the C++ function. Obviously this local variable goes out of scope when the function returns and so the pointer becomes invalid.

Some options:

1. Have the caller locate the string buffer and let the callee populate it.
2. Allocate the returned char* using CoTaskMemAlloc and so meet the expectations of the C# code.
3. Use the COM BSTR type which the pinvoke marshaller does understand.