Search code examples
.netvb.netwindowsvisual-studio-2015registry

Installer custom action can't read all registry values


Using Visual Studio 2015. I'm trying to build an installer custom action that will "unregister" an Excel add-in on uninstall. Essentially, it needs to look through the keys in HKCU\Software\Microsoft\Office and find any subkey that is a version number (16.0 e.g.) then look in the Excel\Options subkey (if it exists) and check for the add-in name in one of the OPEN values (Excel enumerates add-ins in the registry using OPEN, OPEN1, OPEN2, etc).

When I debug my custom action, it looks like it is unable to see all of the registry values. As an example, it reports that there are 8 subkeys under HKCU\Software\Microsoft\Office when there are actually 10 subkeys. I'm guessing that this is due to registry virtualization so I've attempted to force the application to open a specific registry view as below:

x64: RegistryKey.OpenBaseKey(RegistryHive.CurrentUser, RegistryView.Registry64)

x86: RegistryKey.OpenBaseKey(RegistryHive.CurrentUser, RegistryView.Registry32)

Using either of these calls results in the exact same limited registry view (I still see 8 keys rather than 10). So it seems that the installer custom action cannot force a specific registry view for some reason.

I built a console application to do some additional testing around this and the console application can see the full 10 keys regardless of whether it's compiled to target x64 or x86 platforms.

I'm kind of at a loss here. Is this a known issue with VS2015 Setup Projects? Are they limited in their ability to view certain parts of the registry? Or, is it just a code error on my part?

Here is the code that I'm attempting to use to unregister the add-in, if it helps. I added a ton of error checking along the way as it was causing fatal errors in the uninstall process. Uninstall works now (insofar as it doesn't cause a crash) but doesn't actually unregister the add-in since, as noted, it apparently cannot see the full registry.

If Registry.CurrentUser.OpenSubKey("Software\Microsoft\Office", True) IsNot Nothing Then
    Dim regCUOffice As RegistryKey = Registry.CurrentUser.OpenSubKey("Software\Microsoft\Office", True)
    If regCUOffice.GetSubKeyNames.Count > 0 Then
        For Each strKeyName As String In regCUOffice.GetSubKeyNames
            If regCUOffice.OpenSubKey(strKeyName & "\Excel\Options", True) IsNot Nothing Then
                Dim regExcelOptionsKey As RegistryKey = regCUOffice.OpenSubKey(strKeyName, True).OpenSubKey("Excel\Options", True)
                For Each strValueName As String In regExcelOptionsKey.GetValueNames
                    If strValueName IsNot Nothing Then
                        If (strValueName.Equals("OPEN") Or strValueName.StartsWith("OPEN")) Then
                            If regExcelOptionsKey.GetValue(strValueName) IsNot Nothing Then
                                If regExcelOptionsKey.GetValue(strValueName).Equals("MyAddIn.xll") Then
                                    regExcelOptionsKey.DeleteValue(strValueName)
                                End If
                            End If
                        End If
                    End If
                Next
                regExcelOptionsKey.Close()
            End If
        Next
    End If
    regCUOffice.Close()
End If

Solution

  • After a ton of research and work on this, I believe I figured out the issue.

    Apparently, any custom actions built in a setup project in Visual Studio run as a generic SYSTEM account. So, the HKCU registry hive is that of the SYSTEM account and not the currently logged in user. It is possible, however, to circumvent this behavior.

    The most workable solution that I came across involves flipping the impersonate flag in the MSI after it's built. This allows the install custom actions to impersonate the currently logged in user. Unfortunately, flipping this flag is not particularly intuitive. I eventually came across this script which does all the legwork for you:

    // CustomAction_Impersonate.js <msi-file>
    // Performs a post-build fixup of an msi to change all deferred custom actions to Impersonate
    // Constant values from Windows Installer
    var msiOpenDatabaseModeTransact = 1;
    
    var msiViewModifyInsert         = 1
    var msiViewModifyUpdate         = 2
    var msiViewModifyAssign         = 3
    var msiViewModifyReplace        = 4
    var msiViewModifyDelete         = 6
    
    var msidbCustomActionTypeInScript       = 0x00000400;
    var msidbCustomActionTypeNoImpersonate  = 0x00000800
    
    if (WScript.Arguments.Length != 1)
    {
           WScript.StdErr.WriteLine(WScript.ScriptName + " file");
           WScript.Quit(1);
    }
    
    var filespec = WScript.Arguments(0);
    var installer = WScript.CreateObject("WindowsInstaller.Installer");
    var database = installer.OpenDatabase(filespec, msiOpenDatabaseModeTransact);
    
    var sql
    var view
    var record
    
    try
    {
           sql = "SELECT `Action`, `Type`, `Source`, `Target` FROM `CustomAction`";
           view = database.OpenView(sql);
           view.Execute();
           record = view.Fetch();
        //Loop through all the Custom Actions
           while (record)
           {
               if (record.IntegerData(2) & msidbCustomActionTypeInScript)
               {
                   //We must flip the msidbCustomActionTypeNoImpersonate bit only for deferred custom actions
                   record.IntegerData(2) = record.IntegerData(2) & ~msidbCustomActionTypeNoImpersonate;
                  view.Modify(msiViewModifyReplace, record);
               }
            record = view.Fetch();
           }
    
           view.Close();
           database.Commit();
    }
    catch(e)
    {
           WScript.StdErr.WriteLine(e);
           WScript.Quit(1);
    }
    

    Save this script as CustomAction_Impersonate.js in the project folder for your Setup Project (i.e. where your SetupProjectName.vdproj file is located). Then, in Visual Studio, select your Setup Project and open the properties window. In the PostBuildEvent property, add cscript.exe "$(ProjectDir)CustomAction_Impersonate.js" "$(BuiltOuputPath)".

    Basically, this line tells Visual Studio to build your project and, after it's successfully built, run the script that you saved. The script flips the impersonate flag to allow the installer custom action to run as the logged in user.

    In my preliminary testing, this seems to do the trick. I'll update here if I find that this solution is not workable for some reason. Just wanted to share the answer in case anyone else runs across this.