Is it possible to prevent my Sandcastle documentation project from building each time I hit 'Build-->Build Solution'?
This is especially annoying in Debug configuration.
Yes. Go into Build > Configuration Manager
and un-tick build on the Sandcastle project. I usually also do this for the release configuration too.
Then, add a new configuration, named for example 'Sandcastle', and tick all of the projects to build.
This way you only generate your Sandcastle documentation when you explicitly run the appropriate configuration, either through VS or through your CI.