On Windows, some binary files e.g. .obj are in COFF format, and others e.g. .exe and .dll are PE, which is COFF with an extra header. It seems you need to look at the file extension to know which to expect.
Is there a comprehensive list of which file extensions use COFF and which use PE?
I cant comment but to add to Mike Dimmick's answer. Magic Numbers is how the executable file on linux is able to tell file types.
Extensions are used in windows to define default behaviors for parsing a binary. This can get confusing/dangerous, for example, if Program A knows how to handle/parse multiple file types (think Image Preview), a file could have the extension of one type, and be parsed as another. Windows will open Program A feeding it the file. Program A will parse the file and usually disregard the extension all together to determine how to handle the file.
This can be used in order to social engineer (trick) users into opening "familiar" file formats but hit code execution for more obscure formats.