I have a C/C++ DLL. I have and ASP.NET application calling this DLL with P/Invoke. Sometimes one of this calls throws an SEHException.
Right now for testing purposes I call the function with the same data before anything else.
DllFunctions.MyStructure[] structArray = new DllFunctions.MyStructure[2];
structArray[0].A = 10;
structArray[0].B = 10;
structArray[0].C = 15;
structArray[0].D = 15;
structArray[1].A = 8;
structArray[1].B = 12;
structArray[1].C = 13;
structArray[1].D = 17;
int arraySize = 2;
DllFunctions.MyStructure[] otherArray = new DllFunctions.MyStructure[0];
DllFunction.ProblematicFunction(structArray, arraySize, otherArray, 0);
When I start the application in debug mode sometimes it happens to throw me the SEHException, sometimes it doesn't. What can cause random looking behaviour like this?
EDIT: The function runs fine if called from a C++ console application with the same data.
EDIT: the P/Invoke signature
[DllImport("mylib.dll")]
public extern static MyStructure ProblematicFunction(MyStructure[] structs1, int arrayLen1, MyStructure[] structs2, int arrayLen2);
Starting an application in Debug mode disables the Low-Fragmentation Heap - you're still corrupting memory, you're just getting lucky and trashing some padding memory. If you want to see the normal behavior, run the app using Ctrl-F5, then Attach To Process with another VS instance