I'm currently studying different compression algorithms such as huffman, adaptive huffman and Lempel Ziv algorithms but I don't really understand how it should work on a random file.
So I know that they work on text file, but is it the only thing they work on? Can I use Huffman to compress an audio file or an image and if so, how do I know the size of the "blocks" I'll be using for the algorithm ?
Yes, the algorithms that you mention there all work equally well on binary files - it's just for convenience that most papers use character data in their examples.
As for block size, though this is not a requirement, modern general purpose compression algorithms invariably treat the input as a stream of bytes (8-bit values).
Note that while you can in principle attempt to compress an audio file with Huffman compression, the outcome may be unrewarding because Huffman relies on certain symbols being more frequent than others. Special purpose compression algorithms, like MPEGx, are typically used for audio.