Let's say I have designed s very important system, and this system costs thousands dollars. I want to protect my system with a serial number as I know crackers will try to edit the binary code to bypass the serial number.
I have read about using a checksum function and apply it over my binary code and check the value if changed, but again, we are talking about a condition a cracker can avoid by editing the code.
My question is: what's the most used technique to protect important programs?
If your app is working offline, whatever checks you do (check sums, serial code validity, etc), do them often, repeating verification code, in many routines of your software. Obfuscate your code, to make reverse engineering a more difficult task, and, if you have the possibility, implement an online check, part of the core functionality of your app residing on your server, and being serviced only to those installations that you have checked server-side for valid license key. Associate the license key to some form of unique identifier of the hardware the app is running on, and if you check online, have statistics concerning the IPs that make the verification request: if you encounter more IPs trying to verify the same license key, contact the buyer and approve a list of IPs they usually log on from, whilst blacklisting any other until specific request from them, either by mail or by phone.