I have integration tests with MSTest that use WatiN to test my website after every build. I would like to use this same DLL to run nightly (or even hourly) tests against my live site, to ensure everything is still copasetic. However, if there are any errors in this process I would then like to get an email, but I would also like to save the results of each test to be reviewed (probably just a pass-fail report or something).
I could just throw together a Windows service that calls the MSTest EXE directly, but is there something like this already built? Is there a better process I haven't considered?
I have access to separate email servers, web servers, etc., so I should still be notified if my main site goes down ... I just need a way to notify myself.
My test DLL could be easily switched to NUnit, if necessary.
Continuous integration software like Jenkins or CruiseControl.NET is exactly what you're looking for. They are designed to run arbitrary jobs based on triggers such as time of day, a source code commit, periodically, or just about anything else. There are plenty of how-to articles for setting up one of these servers to run the tools you mention.