Search code examples
mathbinaryentropytelecommunication

Entropy in binary files - what's the purpose?


I've read the Shannon's theory to entropy and how to implement such an algorithm for a binary file (pretty easy)

But I can't figure out what's the purpose on calculating the entropy of a .exe or any binary file

I can understand calculating the entropy of a telecommunication message (worst case: every symbol has the same probability) but I can't get the purpose for a binary file.

Someone can explain me this?


Solution

  • Suppose some information or dynamical system or distribution. Entropy is a measure of disorder in that information/system/distribution. If entropy grows disorder grows too. If entropy is 0 it means that there's no disorder, that distribution is constant.

    Applying that all to files I can suppose, that if disorder in it is high file is harder to compress. And files with low entropy (major portion of symbols in that file coincide) are compressed better. Generally, I suppose that files with low entropy will be easier to process.