Search code examples
c#asp.net-coregstreamerheap-corruption

ASP.NET Core app heap corruption with release build


I have an ASP.NET Core web app I've been developing. Recently updated to .NET 8.0. This app interfaces with GStreamer using GStreamer-Sharp bindings. In order to get the bindings to work, I used the .NET upgrade assistant tool to upgrade the binding solution, then build the code and build a nuget package.

When I test in Visual Studio 2022, I'm using debug build configuration. Everything works correctly. I noticed when deploying to production it would get a corrupt heap error in Event Viewer as soon as I hit a certain point in GStreamer code. I decided to test Release build from Visual Studio and it's also crashing. Both Debug and Release builds on my dev machine are utilizing the same GStreamer install, so I think GStreamer itself isn't the issue.

From Event Viewer:

Faulting application name: SecurityCameraViewer.exe, version: 1.0.8852.29269, time stamp: 0x65cd0000
Faulting module name: ntdll.dll, version: 10.0.19041.3996, time stamp: 0x39215800
Exception code: 0xc0000374
Fault offset: 0x00000000000ff349
Faulting process id: 0x9584
Faulting application start time: 0x01da809cae75c5ef
Faulting application path: C:\Users\Dan\source\repos\SecurityCameraViewer\SecurityCameraViewer\bin\Release\net8.0\SecurityCameraViewer.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: 4f9a8b5b-78a9-4ea0-86d1-7a1f7e02115a
Faulting package full name: 
Faulting package-relative application ID: 

Using DebugDiag, I was able to get this:

DetailID = 1
    Count:    1
    Exception #:  0X80000003
    Stack:        
        ntdll!RtlReportCriticalFailure+0x56
        ntdll!RtlpHeapHandleError+0x12
        ntdll!RtlpHpHeapHandleError+0x7a
        ntdll!RtlpLogHeapFailure+0x45
        ntdll!RtlpFreeHeapInternal+0x4e0
        ntdll!RtlFreeHeap+0x51
        ucrtbase!_free_base+0x1b
        gstsdp_1_0_0!gst_sdp_message_free+0x1b
        gstwebrtc_1_0_0!gst_webrtc_session_description_free+0x18
        gobject_2_0_0!g_value_unset+0x29
        gstreamer_1_0_0!gst_structure_free+0x60
        gstreamer_1_0_0!gst_promise_expire+0x163
        gstreamer_1_0_0!gst_mini_object_unref+0x307
        gstwebrtc!gst_plugin_webrtc_register+0x9817
        glib_2_0_0!g_list_sort_with_data+0x231
        glib_2_0_0!g_source_destroy+0xdc
        glib_2_0_0!g_main_depth+0x1ce
        glib_2_0_0!g_main_context_is_owner+0x20a
        glib_2_0_0!g_main_loop_run+0x133
        gstwebrtc!gst_plugin_webrtc_register+0x9fed
        glib_2_0_0!g_thread_new+0x1a2
        glib_2_0_0!g_rw_lock_writer_unlock+0x4a7
        ucrtbase!thread_start<unsigned int (__cdecl*)(void *),1>+0x42
        KERNEL32!BaseThreadInitThunk+0x14
        ntdll!RtlUserThreadStart+0x21


DetailID = 2
    Count:    2
    Exception #:  0XC0000374
    Stack:        
        ntdll!RtlReportFatalFailure+0x9
        ntdll!RtlReportCriticalFailure+0x97
        ntdll!RtlpHeapHandleError+0x12
        ntdll!RtlpHpHeapHandleError+0x7a
        ntdll!RtlpLogHeapFailure+0x45
        ntdll!RtlpFreeHeapInternal+0x4e0
        ntdll!RtlFreeHeap+0x51
        ucrtbase!_free_base+0x1b
        gstsdp_1_0_0!gst_sdp_message_free+0x1b
        gstwebrtc_1_0_0!gst_webrtc_session_description_free+0x18
        gobject_2_0_0!g_value_unset+0x29
        gstreamer_1_0_0!gst_structure_free+0x60
        gstreamer_1_0_0!gst_promise_expire+0x163
        gstreamer_1_0_0!gst_mini_object_unref+0x307
        gstwebrtc!gst_plugin_webrtc_register+0x9817
        glib_2_0_0!g_list_sort_with_data+0x231
        glib_2_0_0!g_source_destroy+0xdc
        glib_2_0_0!g_main_depth+0x1ce
        glib_2_0_0!g_main_context_is_owner+0x20a
        glib_2_0_0!g_main_loop_run+0x133
        gstwebrtc!gst_plugin_webrtc_register+0x9fed
        glib_2_0_0!g_thread_new+0x1a2
        glib_2_0_0!g_rw_lock_writer_unlock+0x4a7
        ucrtbase!thread_start<unsigned int (__cdecl*)(void *),1>+0x42
        KERNEL32!BaseThreadInitThunk+0x14
        ntdll!RtlUserThreadStart+0x21

I'm a bit stuck on how to debug this. I'm not sure what goes on in a release build of my ASP.NET Core project that would affect the GStreamer side of things unless there's something about building the bindings that I need to change.

Edit 1

I've done some more testing, mainly just changing things a bit at random. I'm unsure why I thought release mode was crashing when ran through VS because now it doesn't. However, I've noticed if I change the "Target OS" from "(None)" to "Windows", it will crash regardless of being in debug or release mode.


Solution

  • After discussing the issue with the .NET Platform developers on GitHub, I was able to eventually figure out the issue.

    The GStreamer-Sharp library is both very poorly documented as well as, imho, strange in how it utilizes IDisposables. In my quest to fix memory leaks, I ultimately started resorting to trying to call Dispose() as soon as I thought possible on everything that was IDisposable. Unfortunately this led to some calls that I shouldn't have been making. The end result was I believe what they call a double free error. Just due to luck apparently, the build I was running from VS never failed on me, but my production deployment consistently failed.