When debugging with vs, it tries to load symbols for all dlls.
For some reason, this takes a long time in remote-debugging (although there are no symbols on the target machine).
How can I tell vs not to load symbols for system dlls?
Try going to Debug -> Options and Setting -> Symbols and specifying what modules you want symbols to be automatically loaded for. I'm not sure if that will prevent system dlls from loading though.
There's also a "Just my code" option under Debugging -> General that you might check.
Also, I'd suggest trying to remove all unused references. If you are working on a VB.Net project, this is as easy as opening the project properties and clicking on the unused references under the references tab. Usually there are several system dlls (System.Xml.Linq for example) that VS kindly assumes you'll need that are referenced. There's a good chance they aren't being used (for example, you're never using any of the Linq extension methods) that can be removed.