I thought I'd post this Q&A up because there is very little information about using middleman on windows (I am using windows 7, specifically), and I could not find a solution to this online but was able to figure it out myself.
When booting up the middleman server and receiving:
The Middleman is Standing Watch at http://0.0.0.0:4567
I am unable to view the webpage, simply, nothing at all loads. I turned off the firewall, eliminating it as a potential issue and also tried running the server on a different port. I still got nothing.
0.0.0.0 just stand for "all IPv4 IPs", in this case it means that the middleman server listens on port 4567 on all local IPs.
It's important to note that 0.0.0.0 is not an actual IP you can use in your browser. Instead, you can navigate to any of your actually assigned local IPs, e.g. 127.0.0.1 which is the local IP on your system.
Just using localhost doesn't work as Windows has IPv6 enabled by default (which has precedence before IPv4) and thus maps localhost to ::1, the local IPv6 IP. As Middleman only binds to the local IPv4 IPs, it doesn't answer on ::1.