Here's the documentation article for SignTool: SignTool.
I'm not really sure where should I specify the SignTool parameters and special sequences. Is it in the [Setup] section or in the compiler IDE (via the Tools | Configure Sign Tools... menu) or both? If I have specified all the parameters in the compiler IDE, do I have to do that again in the [Setup] section? Also, I don't really understand this part:
byparam=$p
Note: for security reasons you should give a unique name to any Sign Tool set to $p, and not use a byparam name copied from this example. Consider what happens if you #include a third-party file that says:
[Setup] SignTool=byparam format c:
How does that $p sequence really work here?
In the SignTool directive, you specify, which sign tool definition you want to use in that particular script. The tool can be defined in IDE or using /S compiler commandline switch.
The SignTool directive can contain additional parameters, which can be referred to by the definition using the $p special sequence. That allows you the flexibility to define part of the command in the script and part in the definition. The choice is completely yours. To be honest, I do not really understand the reason for this, as the same flexibility can be achieved using Inno Setup preprocessor. There might be some historical reason. I do not know if the SignTool predates the built-in preprocessor, or if it was the other way around.
Anyway, the above means that these three are equivalent:
Everything in the definition
Definition:
mysigntool=signtool.exe sign /a /n $qMy Common Name$q /t http://... $f
Script:
[Setup]
signtool=mysigntool
Part in the definition, part in the script
Definition:
mysigntool=signtool.exe sign /a $p
Script:
[Setup]
signtool=mysigntool /n $qMy Common Name$q /t http://... $f
Everything in the script
Definition:
anysigntool=$p
Script:
[Setup]
signtool=anysigntool signtool.exe sign /a /n $qMy Common Name$q /t http://... $f