Assigning same User Application Model ID to WinForms app and VB6 app

We have a mixed VB6 application with some .NET codebase. The VB6 project is nearing capacity, so I'm migrating the main application to a WinForms application. Because of the amount of legacy codebase and available resources, it is not possible to migrate it all to the .NET application. We plan to keep the VB6 project as a second process for access to those forms we will not be migrating now, using named pipes for IPC.

Right now, I'm trying to group the two processes on the task bar. Everything I've found points to using the Win32 API's SetCurrentProcessExplicitAppUserModelID() to give both processes the same AUMID.

In the .NET application, I have:

[DllImport("shell32.dll", SetLastError = true)]
static extern int SetCurrentProcessExplicitAppUserModelID([MarshalAs(UnmanagedType.LPWStr), In] string AppID);

/// <summary>
/// The main entry point for the application.
/// </summary>
[STAThread]
static void Main()
{
    SetCurrentProcessExplicitAppUserModelID("MyCompany.MyApplication");
    // Other presumably irrelevant initialization
}

The .NET application starts the VB6 application using a Process.Start as part of its initialization. In the VB6 application, I have:

Private Declare Function SetCurrentProcessExplicitAppUserModelID Lib "shell32.dll" (appID As Long) As Long

Private Sub Main()
  Dim strAUMID As String
  strAUMID = "MyCompany.MyApplication"
  SetCurrentProcessExplicitAppUserModelID StrPtr(strAUMID)
  ' Other presumably irrelevant initialization
End Sub

I expect the forms from both processes would be grouped under a single icon on the taskbar with these changes, but they remain separate by process. I've tried to add some diagnostic code (which I omitted above) to try to get to the bottom of it.

It appears that the .NET application is working as expected. When I use GetCurrentProcessExplicitAppUserModelID() it outputs "MyCompany.MyApplication".

But, when I use the Get in the VB6 application, it outputs something else. The Get in VB6 will return a StrPtr instead of a String. I found code at https://www.vbforums.com/showthread.php?682690-Using-the-results-of-StrPtr to convert the StrPtr to the corresponding String, and it doesn't convert back to "MyCompany.MyApplication".

I'm not sure what else to try.

Edit: This is the current code in the .NET application.

       [DllImport("shell32.dll", SetLastError = true)]
static extern int SetCurrentProcessExplicitAppUserModelID([MarshalAs(UnmanagedType.LPWStr), In] string AppID);

[DllImport("shell32.dll", SetLastError = true)]
static extern int GetCurrentProcessExplicitAppUserModelID([MarshalAs(UnmanagedType.LPWStr), Out] out string AppID);

[STAThread]
static void Main()
{
    var result = SetCurrentProcessExplicitAppUserModelID("MyCompany.MyApplication");
    MessageBox.Show(result.ToString());
    string appId;
    result = GetCurrentProcessExplicitAppUserModelID(out appId);
    MessageBox.Show($"{result}: {appId}");
    // Other code
}

Both Set and Get return 0, and the appID is MyCompany.MyApplication.

This is what was in the VB6 application.

Private Declare Function SetCurrentProcessExplicitAppUserModelID Lib "shell32.dll" (appID As Long) As Long
Private Declare Function GetCurrentProcessExplicitAppUserModelID Lib "shell32.dll" (ByRef appID As Long) As Long
Private Declare Function SysAllocString Lib "OleAut32" (ByVal strPtr As Long) As String

Private Sub Main()
  Dim ret As Long
  Dim appID As Long
  Dim appString As String
  Dim strAUMID As String
  strAUMID = "MyCompany.MyApplication"
  ret = SetCurrentProcessExplicitAppUserModelID(StrPtr(strAUMID))
  MsgBox CStr(ret)
  ret = GetCurrentProcessExplicitAppUserModelID(appID)
  appString = StrConv(SysAllocString(appID), vbFromUnicode)
  MsgBox CStr(ret) + ": " + appString
  ' Other code
End Sub

Both Get and Set returned 0, appString displayed as "??".

Solution

Looks like your Declare definition is wrong. The parameter of SetCurrentProcessExplicitAppUserModelID needs to be ByVal because the default in old VBA/VB6 is ByRef.

Also you need to free the pointer you get from GetCurrentProcessExplicitAppUserModelID using CoTaskMemFree.

Private Declare Function SetCurrentProcessExplicitAppUserModelID Lib "shell32.dll" (ByVal appID As Long) As Long
Private Declare Function GetCurrentProcessExplicitAppUserModelID Lib "shell32.dll" (appID As Long) As Long
Private Declare Function SysAllocString Lib "OleAut32" (ByVal strPtr As Long) As Str
Private Declare Sub CoTaskMemFree Lib "Ole32" (ByVal strPtr As Long)

Private Sub Main()
  Dim ret As Long
  Dim appID As Long
  Dim appString As String
  Dim strAUMID As String
  strAUMID = "MyCompany.MyApplication"

  ret = SetCurrentProcessExplicitAppUserModelID(StrPtr(strAUMID))
  MsgBox CStr(ret)

  ret = GetCurrentProcessExplicitAppUserModelID(appID)
  appString = StrConv(SysAllocString(appID), vbFromUnicode)
  If ret = 0 Then CoTaskMemFree(appID)
  MsgBox CStr(ret) + ": " + appString

  ' Other code
End Sub