I'm am completely new to Emacs, so this may be a very naive question but I can't seem to find anything on this.
The basic issue seems to be that I can't open any of the "Info" topics.
The error that I get when I try to open any of them is:
Uncompression program 'gzip' not found
I have installed Gzip for Windows (here), but that did not fix the issue.
I have tried this on both Windows 7 and Windows 8.1 machines. I am using Emacs 24.3 (i386-mingw-nt6.1.7601), although I've also tried Emacs 24.4.50.1 (x86_64-w64-mingw32) from here
The problem is probably that the gzip library isn't in the path that Emacs uses.
Third-party builds of Emacs are difficult to support, and largely unnecessary today. There are now official binary releases of Emacs for Windows that should bundle everything you need. I've had plenty of success using these releases on Windows 7.
Download the latest zip (currently emacs-24.4-bin-i686-pc-mingw32.zip), unzip the file, and run wbin/runemacs.exe. That version should fully support info.