The problem of LibraryImport automatically exiting in .NET 9

I have an existing .NET 8 project that has been running fine and I want to upgrade it to .NET 9.

I need to reference a C++ dll file in the project, and I have always referenced it this way:

  [LibraryImport("Karel")]
  private static partial uint VikeyFind(ref uint pdwCount);

When I debug my program in Visual Studio, my program automatically exits without any errors.

I tried to run the code step by step and found that when the program runs to this point.

 [global::System.CodeDom.Compiler.GeneratedCodeAttribute("Microsoft.Interop.LibraryImportGenerator", "9.0.11.2809")]
 [global::System.Runtime.CompilerServices.SkipLocalsInitAttribute]
 private static partial uint VikeyFind(ref uint pdwCount)
 {
     uint __retVal;

     // Pin - Pin data in preparation for calling the P/Invoke.
     fixed (uint* __pdwCount_native = &pdwCount)
     {
         __retVal = __PInvoke(__pdwCount_native);
     }

     return __retVal;
     // Local P/Invoke
     [global::System.Runtime.InteropServices.DllImportAttribute("Karel", EntryPoint = "VikeyFind", ExactSpelling = true)]
     static extern unsafe uint __PInvoke(uint* __pdwCount_native);
 }

When the line of code

__retVal = __PInvoke(__pdwCount_native);

is executed, the program will automatically exit.

I don't know what the problem is. I tried rolling back the version to .NET 8 and the program ran fine.

I'm not sure if this is a bug or if there are any changes to LibraryImport in .NET 9. Could you please tell me what the problem is?

PS: my .NET 9 SDK version is 9.0.100. What's more, I never change any code for upgrade .NET 9 but only update NuGet Package to latest version. However, after I rolled back to .NET 8 with all the latest version NuGet Package, the program still ran fine.

Solution

It would seems like this breaking change break you program:

https://learn.microsoft.com/en-us/dotnet/core/compatibility/interop/9.0/cet-support

The doc says to add this <CETCompat>false</CETCompat> to .csproj should fix it

edited: a bit more explain:.net 9 add more support for Intel CET(Control-flow Enforcement), which basically add checks for return addresses so to prevent Return Oriented Programming (also known as ROP) and Jump Oriented Programming (also known as JOP) attacks.

Hence dll that is not compiled with Intel CET—compatible options will still attempt to do things that's not allowed by new standard, which will cause process to terminated, according to the docs you can opt out of CET by adding <CETCompat>false</CETCompat>. But if you have access to the source code of that dll, maybe consider recompile it with /CETCOMPAT enabled?