I have a Wix installer to install my visual studio extension (vsix) + related files. However, when I tick the "This VSIX is installed by Windows Installer" cheeckbox in the vsixmanifest under "Install Targets" (this adds the attribute "InstalledByMsi" to the tag and sets its value to "true") the installer exits and I find the following error in the log:
MSI (s) (7C:2C) [12:47:13:966]: Executing op: CustomActionSchedule(Action=viuD024AF088F5BE8AD5BD310C0F79AE3DF,ActionType=1074,Source=C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\VSIXInstaller.exe,Target=/q "C:\Users\me\AppData\Local\My Company\MyPlugin\MyPlugin.vsix",)
CustomAction viuD024AF088F5BE8AD5BD310C0F79AE3DF returned actual error code 1009 (note this may not be 100% accurate if translation happened inside sandbox)
A quick google shows me that error 1009 from VSIXInstaller.exe is called InstallByMsiException.
It works absolutely fine with this checkbox unticked, but it means that visual studio allows users to uninstall the extension via the extensions manager, which won't include the files outside the vsix or the registry entries created by the windows installer.
Is there some setting somewhere I've missed? What can I look for?
So it turns out that if you're installing a vsix file via an msi installer, you are expected to extract the files yourself and manually place them. This works fine if you install for all users, but I'm not sure what to do in the event that you want to install to AppData, since the extension paths appear to be auto-generated in that instance.
The fact that the Wix Toolkit has a nice wrapper function for installing vsix files is misleading since it will not automate the unpacking of a vsix file and cannot install a vsix with the InstalledByMsi attribute set