How do I call this c function in c# (unmarshalling return struct)?

Question

I want to use c# interop to call a function from a dll written in c. I have the header files. Take a look at this:

enum CTMBeginTransactionError {
    CTM_BEGIN_TRX_SUCCESS = 0,
    CTM_BEGIN_TRX_ERROR_ALREADY_IN_PROGRESS,
    CTM_BEGIN_TRX_ERROR_NOT_CONNECTED
};

#pragma pack(push)
#pragma pack(1)
struct CTMBeginTransactionResult {
    char *                        szTransactionID;
    enum CTMBeginTransactionError error;
};

struct CTMBeginTransactionResult ctm_begin_customer_transaction(const char * szTransactionID);

How do I call ctm_begin_customer_transaction from c#. The const char * mapps well to string, but despite various attempts (looking at stackoverflow and other sites), I fail to marshal the return structure. If I define the function to return IntPtr it works ok...

Edit I changed the return type to IntPtr and use: CTMBeginTransactionResult structure = (CTMBeginTransactionResult)Marshal.PtrToStructure(ptr, typeof(CTMBeginTransactionResult)); but it throws AccessViolationException

I also tried:

IntPtr ptr = Transactions.ctm_begin_customer_transaction("");
int size = 50;
byte[] byteArray = new byte[size];
Marshal.Copy(ptr, byteArray, 0, size);
string stringData = Encoding.ASCII.GetString(byteArray);

stringData == "70e3589b-2de0-4d1e-978d-55e22225be95\0\"\0\0\a\0\0\b\b?" at this point. The "70e3589b-2de0-4d1e-978d-55e22225be95" is the szTransactionID from the struct. Where is the Enum? Is it the next byte?

Answer 1

I hate to answer my own question, but I found the solution to marshal the resulting struct. The struct is 8 bytes long (4 bytes for the char * and 4 bytes for enum). Marshalling the string does not work automatically, but the following works:

// Native (unmanaged)
public enum CTMBeginTransactionError
{
    CTM_BEGIN_TRX_SUCCESS = 0,
    CTM_BEGIN_TRX_ERROR_ALREADY_IN_PROGRESS,
    CTM_BEGIN_TRX_ERROR_NOT_CONNECTED
};

// Native (unmanaged)
[StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Ansi)]
internal struct CTMBeginTransactionResult
{
    public IntPtr szTransactionID;
    public CTMBeginTransactionError error;
};

// Managed wrapper around native struct
public class BeginTransactionResult
{
    public string TransactionID;
    public CTMBeginTransactionError Error;

    internal BeginTransactionResult(CTMBeginTransactionResult nativeStruct)
    {
        // Manually marshal the string
        if (nativeStruct.szTransactionID == IntPtr.Zero) this.TransactionID = "";
        else this.TransactionID = Marshal.PtrToStringAnsi(nativeStruct.szTransactionID);

        this.Error = nativeStruct.error;
    }
}

[DllImport("libctmclient-0.dll")]
internal static extern CTMBeginTransactionResult ctm_begin_customer_transaction(string ptr);

public static BeginTransactionResult BeginCustomerTransaction(string transactionId)
{
    CTMBeginTransactionResult nativeResult = Transactions.ctm_begin_customer_transaction(transactionId);
    return new BeginTransactionResult(nativeResult);
}

The code works, but I still need to investigate, if calling the unmanaged code results in memory leaks.