We have a Visual Studio extension, currently installed with MSI, since we need to perform certain tasks after the installation (ngen, registering certain COM servers, etc). In the end of the installation, we run devenv.exe /setup (more specifically, devenv.exe /setup /nosetupvstemplates to make it a bit faster) to register the extension in Visual Studio 2012 and above (in 2010 this step was not required, since VS2010 would rebuild the extensions cache on every startup).
We're currently looking to move to a partial VSIX installation, but there are certain issues:
We have a certain folder structure (additional bundled tools). I wasn't able to find an option to include arbitrary folders and files with in VSIX package. Is this possible?
We're trying to avoid using devenv.exe /setup, since in rare cases, this may cause extensions not to load (as detailed by Remco Mulder on the MSDN forums).
We still need to perform "post-installation" operations, currently not possible with VSIX.
Due to previous 'sins', we hardcoded some of the paths to use the CompanyName\ProductName\Version format that the VSIX packages used to follow, but it seems that VSIX randomy generate a folder name now (as Jason mentions in his answer below).
So what I'd essentially like is to be able to install an "unpacked" VSIX package - have the MSI installer create the file structure on disk (in the correct location, %VSInstallDir%\Common7\IDE\Extensions\Company\Product\Version), and then somehow to have VSIX consume the .vsixmanifest file to register the extension in Visual Studio.
(Initial digging into the Extension Manager API shows there's a method called CreateInstalledExtension in ExtensionManagerService that takes in a path to vsixmanifest, in assembly Microsoft.VisualStudio.ExtensionManager.Implementation.dll, but unfortunately it is internal).
So, without resorting to API hackery, is there any alternative way to essentially install an extension in Visual Studio (2012 and above), without using devenv /setup?
As Jason (and everyone else) suggests - yes, the "correct" (supported) way of registering a Visual Studio package externally is by running devenv.exe /setup, and for the most part - it works without problems (albeit slow, due to entire Visual Studio configuration being rebuilt).
This is a commercial product, and every once in a while we get a support case, complaining that after installing our extension, something bad happened in Visual Studio - other extensions failed to load, problem with Solution explorer, etc. Those cases are rare, but they do happen.
In an attempt to minimize the impact installing our extension does to the user's machine, I was trying to find a suitable way to use the VSIX installation mechanism that does not use devenv.exe /setup, but unfortunately, there isn't a supported scenario that handles all the issues I've raised in the question above.
After extensive research into the problem, I found an undocumented, unsupported solution that solves the problem completely! An empty file, called extensions.configurationchanged, located under %VSInstallDir%\Common7\IDE\Extensions gets touched every time a VSIX package is installed or uninstalled, causing Visual Studio to refresh it's package cache (only!) on next restart! With modifying this file's "last modified date", it will essentially cause Visual Studio to act as if a VSIX package was just installed or uninstalled.
A few quick experiments of copying the extension files to an empty hive, and then touching the extensions.configurationchanged will cause Visual Studio to load the extension (and process all its .pkgdef files) on next restart - no devenv /setup necessary!
Several commercial products offer this solution in the Troubleshooting section of their online help, dealing with loading failure of their extension, so this solution might be the "lesser evil" I was looking for.
Use at your own risk.