I am trying to register an exe to a custom protocol using a packaging project. I have added the following code in appxmanifest of packaging project.
<Extensions>
<uap:Extension Category="windows.protocol" Executable="Assets\app.exe">
<uap:Protocol Name="app.custom.protocol" />
</uap:Extension>
</Extensions>
But I am unable to launch the exe from uwp app using launchUriAsync method. It just shows the dialog asking for an app to open the protocol.
To register a custom protocol for an EXE in your package it needs to be declared as in your appxmanifest. Note that you can have multiple elements in your package. So for example if your main app is a UWP and you also have a Win32 executable (WPF) that you want to register as custom protocol handler you would have to author the package manifest like the below. Note that setting AppListEntry=None will prevent the Win32 EXE from showing up in the start menu.
<Applications>
<Application Id="UWP" Executable="Uwp.exe" EntryPoint="Uwp.App">
<uap:VisualElements … />
</Application>
<Application Id="WPF" Executable="WpfApp2\WpfApp2.exe" EntryPoint="Windows.FullTrustApplication">
<uap:VisualElements AppListEntry="none" … />
<Extensions>
<uap3:Extension Category="windows.protocol">
<uap3:Protocol Name="app.custom.protocol" />
</uap3:Extension>
</Extensions>
</Application>
</Applications>
However, if all you want to do is launch your Win32 EXE from your UWP you will be better off using the FullTrustProcessLauncher API, instead of doing protocol launch:
https://learn.microsoft.com/en-us/uwp/api/Windows.ApplicationModel.FullTrustProcessLauncher