Is there any way I can go back and install a older version of Cygwin?
Say I want the 1.7.9 version, but the setup.exe in the Cygwin website always point to the latest release?
http://www.cygwin.com/setup.exe will always point to the latest release. The only previous release available at cygwin.com is the 'setup-legacy' file, which is version 1.5.25 and is compatible with older versions of Windows. Downloading older versions of Cygwin is discouraged because of incompatibility with the latest available packages.
That said, if you are certain you want an older version of setup.exe, the only way to get it would be to find the file mirrored elsewhere. Simply google the specific version you want, and you should be able to easily find what you are looking for. download.cnet.com, for example, has many previous releases: see for yourself.
In summary, there is no 'official' way to get previous Cygwin releases, so you will have to find a mirror of that specific release.