I've been trying to understand what exactly Cygwin and MinGW are and find it very confusing. If you want to program in C++ don't you just need a C++ compiler? From what I've read it seems like they strive to provide a Unix like operating system on Windows but I don't see the significance of this. I mean is there some reason that C++ can't natively be compiled on Windows? I also read that they contain libraries, is this because the C++ core language doesn't support many functions so very common libraries such as math.h come with it?
This explanation is bound to be already written down elsewhere, but here goes anyway...
Cygwin is a POSIX compatible runtime built on top of the Win32 API. It provides a largely compliant POSIX C library, and around that, there is a sort of "Cygwin Distro" with package manager that allows you to install a ton of Unix programs ported to run under Cygwin. These include a Unix shell, GCC (for Cygwin), X, a bunch of cross-compilers, etc...
MinGW(-w64) are projects providing Free (as in Public Domain) Win32 headers and libraries. You can see this as a Free replacement for the headers and libraries in the Windows SDK (of old). These can be used in conjunction with GCC. Both projects and other third parties provide native GCC builds for Windows, which do not need Cygwin at all. These packages can be seen as a "full" (to the extent of completeness of MinGW(-w64)) replacement for the Windows SDK.
Note that Cygwin also provides Cygwin to Win32 cross-compilers: these run on the Cygwin platform but produce native Win32 executables.
Another player, MSYS, is a lightweight fork of Cygwin, which provides only the minimum amount of tools to run autotools build scripts on Windows. MSYS must be used in conjunction with the native MinGW(-w64) tools, it is not Cygwin and you cannot easily extend MSYS.
So "MinGW(-w64)" often refers to the whole Free Windows GNU Toolchain (native or a cross-compiler running on top of Cygwin), although strictly speaking it is only the headers and libraries.
To use GCC's C++ compiler, I suggest you use either my packages (to create 32-bit and 64-bit executables) or mingw-builds. Alternatively, you can install Cygwin and use its setup.exe to install the MinGW-w64 cross-compiler and use it from there, but if you do not need near-perfect Unix emulation for build scripts, I'd suggest against it.
EDIT You may or may not be aware that there is another alternative, the old Interix, which sidesteps the Win32 subsystem and builds directly on top of the NT kernel. This is provided as Windows Services for Unix or Subsystem for Unix Applications (available for Windows 7 and Windows 8 and older versions of Windows). This solution is available for Windows Pro/Ultimate version customers and provides you with the closest to native Unix support you are going to get. Heck, you can even use Visual Studio to debug your Unix software :)