Recently been trying to setup a build server to help automate my deployment.
My question is are people using build servers like TeamCity to push changes to production (WebDeploy)servers (as well as your test server)? If so are you doing it by rebuild from source control each time or using the build files you pushed to your test server (would have also had to create all web.configs at the time you did build for test server).
It is generally a good idea to use the same set of build files you generated for your test environment deploy as for your prod deploy, as this is the only way you can guarantee that what went into the build will be consistent (i.e. in between the two builds something could have been installed on the build server or your VCS may return a slightly different set of source code files).
This is best achieved in TeamCity with the use of build artifacts, which harvest the output of your builds into a special storage area, allowing them to be drawn down in another build configuration for later use. More info in the TeamCity docs http://confluence.jetbrains.net/display/TCD6/Build+Artifact
As far as your configs go, consider using config transforms. These allow you to have a variant of your Web.config for each target environment, with different connection strings, environment constants etc. More info on MSDN http://blogs.msdn.com/b/webdevtools/archive/2009/05/04/web-deployment-web-config-transformation.aspx
If you actually intend on using TeamCity to deploy to your Production environment, you may want to consider locking these down using TeamCity permissions so that only a restricted group has access to run those build configurations (in our shop this is actually a requirement to ensure seggregation of duties between Dev and Ops teams). As an added precaution, set up a separate build agent just for your Prod builds and keep it disabled until you need it (prevents you from accidentally Running a prod build configuration... yep I've been there sigh).