Search code examples
.netwinapivb6interoppinvoke

SetWindowLong Hanging

Why does SetWindowLong(myForm.hWnd, GWL_HWNDPARENT, parentHwnd) hang?

I can recreate this problem consistently doing these three steps.

  1. Create the .NET Form
  2. Initalize the WaitWindow COM object, call ShowWindow on the COM object while passing the .NET Forms Handle
  3. In VB6 invoke the SetWindowLong method

C# Windows Application (Hangs)

private static void Main(string[] args)
{
      Form form = new Form();
      form.Show();

      Interop.WaitWindow waitWindow = new Interop.WaitWindow();
      waitWindow.ShowWindow(form.Handle.ToInt32(), Language.RISEnglish);
}

C# Console Application (Doesn't Hang)

private static void Main(string[] args)
{
      IntPtr handle = Process.GetCurrentProcess().MainWindowHandle;     

      Interop.WaitWindow waitWindow = new Interop.WaitWindow();
      waitWindow.ShowWindow(handle.ToInt32(), Language.RISEnglish);
}

VB6 Code Snippet

Public Sub ShowWindow(ByVal parentHwnd As Long, ByVal language As Language)

    SetWindowLong(myForm.hWnd, GWL_HWNDPARENT, parentHwnd)  'Hangs Here
    CenterWindow (parentHwnd)

    myForm.ShowRetrieving (language)
    myForm.Show (vbModal)
End Sub

Really would appreciate your help :)

EDIT

I do understand that SetWIndowLong shouldn't be called to change the parent but I'm trying to understand why it hangs only when a .NET form handle is used.

EDIT2

I now believe the issue is not related to SetWindowLong but the actual handle itself. I'm still investigating but it appears that when I call the VB6 code from .NET it creates an RPC thread. I'm not sure yet but I have a feeling it has something to do with a cross-threading issue.

Solution

  • I managed to figure out exactly what was going on and how to fix the problem. I didn't specify my main entry point with the [STAThread] attribute so it was defaulting to MTA instead. This meant when I called the VB6 code it created a RPC Callback Thread and didn't marshall the call to the main thread where the UI executes.

    Peter Mortensen wrote a good explanation about this:

    The STA model is used for COM objects that are not thread safe. That means they do not handle their own synchronization. A common use of this is a UI component. So if another thread needs to interact with the object (such as pushing a button in a form) then the message is marshalled onto the STA thread. The windows forms message pumping system is an example of this.