We have some .NET code that does a call out to a COM component (dnsapi.dll) to query whether a domain has MX records associated with it to validate email addresses entered into our site.
The problem is we've upgrading from 32 bit systems to 64 bit systems and the code that marshals between the COM types and .NET types (and cleans up after) doesn't work properly any more.
The COM call puts the results in memory and returns a pointer to where they are. The code that marshals the results into a .NET struct still works fine but the code that cleans up the memory after corrupts the heap and crashes the IIS app pool.
It's definitely a 64/32 bit problem because I created a small console app that calls the same code and it works fine when I compile it for 32 bit but displays the same heap corruption behavior when I compile it for 64 bit.
I found a stackoverflow post of someone having the same problem and they said the solution was:
The solution is try to make sure the win32 struct and c# struct is bit(bit size) mapping. This can be achieved by using exact c# type for win32 type.
I've not done much interop before and am not sure what needs to change. Here's what the MSDN documentation says the native structure looks like:
typedef struct _DnsRecord {
DNS_RECORD *pNext;
PWSTR pName;
WORD wType;
WORD wDataLength;
union {
DWORD DW;
DNS_RECORD_FLAGS S;
} Flags;
DWORD dwTtl;
DWORD dwReserved;
union {
DNS_A_DATA A;
DNS_SOA_DATA SOA, Soa;
DNS_PTR_DATA PTR, Ptr, NS, Ns, CNAME, Cname, DNAME, Dname, MB, Mb, MD, Md, MF, Mf, MG, Mg, MR, Mr;
DNS_MINFO_DATA MINFO, Minfo, RP, Rp;
DNS_MX_DATA MX, Mx, AFSDB, Afsdb, RT, Rt;
DNS_TXT_DATA HINFO, Hinfo, ISDN, Isdn, TXT, Txt, X25;
DNS_NULL_DATA Null;
DNS_WKS_DATA WKS, Wks;
DNS_AAAA_DATA AAAA;
DNS_KEY_DATA KEY, Key;
DNS_SIG_DATA SIG, Sig;
DNS_ATMA_DATA ATMA, Atma;
DNS_NXT_DATA NXT, Nxt;
DNS_SRV_DATA SRV, Srv;
DNS_NAPTR_DATA NAPTR, Naptr;
DNS_OPT_DATA OPT, Opt;
DNS_DS_DATA DS, Ds;
DNS_RRSIG_DATA RRSIG, Rrsig;
DNS_NSEC_DATA NSEC, Nsec;
DNS_DNSKEY_DATA DNSKEY, Dnskey;
DNS_TKEY_DATA TKEY, Tkey;
DNS_TSIG_DATA TSIG, Tsig;
DNS_WINS_DATA WINS, Wins;
DNS_WINSR_DATA WINSR, WinsR, NBSTAT, Nbstat;
DNS_DHCID_DATA DHCID;
} Data;
} DNS_RECORD, *PDNS_RECORD;
Here's our .NET struct:
[StructLayout(LayoutKind.Sequential)]
private struct DNSRecord
{
public IntPtr pNext;
public string pName;
public Int16 wType;
public Int16 wDataLength;
public Int32 dwflags;
public Int32 dwTtl;
public Int32 dwReserved;
public IntPtr pNameExchange;
public Int16 wPreference;
public Int16 Pad;
}
I know the IntPtr is a different size depending on whether the app is running as 64 bit or 32 bit but that doesn't seem to be something I can specifically control. I'm not sure whether I need to make the fields bigger or smaller. Ideally I'd really like to understand what's going on and why this is a problem.
I know another option would be to just use a 3rd party library that does this purely through CLR code. Unfortunately this is a .NET1.1 library that will be difficult to upgrade right now so we can't just plug in a .NET library. I've had a look for a .NET library that can query MX records and they all require a newer version of .NET.
Yes, your structure is far too short, you didn't handle the Data field correctly. The structure needs to be 64 bytes in 32-bit mode, 88 bytes in 64-bit mode. Memory is getting corrupted in the 32-bit version of IIS as well, you just haven't noticed it.
Use Marshal.SizeOf(typeof(DNSRecord)) to verify. You get the required sizes by adding these fields to the structure:
[StructLayout(LayoutKind.Sequential)]
struct DNSRecord {
public IntPtr pNext;
// etc..
public Int16 Pad;
private int Pad1;
private int Pad2;
private int Pad3;
private int Pad4;
private int Pad5;
private IntPtr Pad6;
private IntPtr Pad7;
private IntPtr Pad8;
}