We've been experiencing a strange issue with SQL 2008 R2 (10.50.1600) installed as a named instance. In order for any external clients to connect, we have a certain procedure we have to follow, but should not have to. Now I did in fact open the TCP/IP and Named Pipes protocols on the SQL server and restarted it, this isn't the problem. We're on an Active Directory Domain (running from Server 2003). The problem exists no matter what OS the server or client is (XP, 2003, 2008, Vista, 7, 64bit, 32bit, etc.). The problem also persists from anything which can connect, for example, SQL Management Studio, ADO (from our applications), etc.
The problem is that before any client can connect to this server, each client machine must first connect to this server through ODBC (and we don't use ODBC). Any attempt to connect to a 10.5 SQL server before doing this results in "Server does not exist or access denied". But once we can connect in the ODBC (through Named Pipes), then everything else starts to work. The same issue occurs both when using the Computer Name and IP Address. In fact, if we want to connect with computer name \ instance name, then we have to do so first in the ODBC, and then if we want to connect via the IP address \ instance name, then we have to do the same also for that.
We've been having to do this on every single client computer. Again, once the ODBC is able to connect to this SQL server through Named Pipes, then all future attempts from that client work.
What could be causing this to occur? How to avoid it? I should not have to do this "ODBC Trick" as we've been calling it. I've never had this issue on any other version of SQL.
The issue might be related to the SQL Browser service. Each sql instance will have a different port number - try connecting from the client as IP Address,Port (e.g. 123.123.123.1,1433) - this will exclude DNS and Browser from the equation
Edit: now knowing that it is browser related, try see why clients can't access SQL Browser (usually Port 1434). Service not started? Possibly firewall blocking?
Microsoft have tied down everything security wise now by default, so any new configuration now generally requires quite a bit of detailed security planning, policy configuration, permissions etc. Welcome to the age of non-trust ;)