I have a COM component written in VB6 that I'm using from a Visual Basic .NET application through COM interop.
There is a method that throws an OutOfMemoryException when called from VB.NET. However, when I run the exact same call from a Visual Basic 6 application, everything works fine.
I cannot post code here because it is too long and hard to follow (I'm tasked with fixing it) and I can't pinpoint the problem, because when I try to debug it from VB6, the error doesn't show up.
What could be causing this distinct behavior?
This was working fine until I fixed a performance problem using a Dictionary object (a COM one from the Microsoft Scripting Runtime). Unless the Dictionary is growing irrationally or leaking, I don't see how it could be causing this, since it never grows past 100-200 elements and only one is created before the exception is thrown.
I tried the same fix with a Collection object and an awkward Exists() function and the same problem happens. If I bail out early by returning Nothing, it works (i.e., it throws NullReferenceException, as would be expected).
Memory management in VB.NET is very different. VB6 uses reference counting to release COM objects, VB.NET uses the garbage collector. You can get into a situation, especially when you are unit-testing the component, where the GC doesn't run frequently enough to release the COM objects. Those objects allocate unmanaged memory, that doesn't put pressure on the garbage collector to kick off a collection.
Start diagnosing this with Perfmon.exe, Performance Monitor. Right-click the graph, Add Counters and select .NET CLR Memory, # Gen 0 collections and pick your process. Verify that you see the counter incrementing at a reasonable rate, at least one a second.
Fixing such a problem is fairly unpleasant, make sure that this is not an artificial problem that is induced by isolating the component. Or a plain bug in your code, keeping a reference to the COM object. Calling GC.Collect() followed by GC.WaitForPendingFinalizers() will release COM objects. So does calling Marshal.ReleaseComObject(). Using GC.AddMemoryPressure() is a possible workaround as well, but you do have to have a reasonable idea how much unmanaged memory the COM object requires.