I am a new developer working on an open source application and I am trying to figure out how one signs a program.
I understand why code signing is a thing, and I know how to get a certificate (from a CA or individually), but I can't find anything explaining the actual process of signing itself once you have your private key.
How does one sign code with a signing certificate? Specifically, do you sign the source code, the built executable, something else entirely? Do you run a specific app, or include something in the code?
related but discrete question: What to do with the code signing certificate?
You have a code signing certificate which contains a private key and a public key.
You calculate a cryptographically save checksum of your code and data, encrypt it with your private key. You add the encrypted checksum and your public key to your code.
Now anyone can calculate the same checksum of your code, decrypt the encrypted checksum with the public key, and compare them. If they match, then it is proven that the code was supplied by someone who has your private key. If they don't match, then it is proven that this code is not the same as the on of which you encrypted the checksum.
To be useful, the operating system would perform this kind of test before any application is allowed to run. And the public key would be signed with well known private key owned by the OS supplier, and would likely be accompanied by data that can be used to identify the owner of the public key.
Obviously EvilHacker could create malware and code sign it. So the fact that it is signed is meaningless. But you would also have the identity of EvilHacker, or the operating system wouldn't start the application. So the operating system will never allow you to run a hacked application or one where the owner of the public key is not known to the creator of the OS.
The second worst a hacker can do successfully is to completely remove the code signing and give you unsigned hacked code - the OS will likely ask you whether you want to run unsigned code or not, or not allow it at all.
The worst a hacker can do is either to con the os owner to accept EvilHacker's code signing certificate, or steal a code signing certificate and create malware that is actually correctly signed.