I'd like to know if a connected socket is using telnet or a different protocol.
What is required to know this? Do I need to inspect network traffic?
Also is there a third party library that might help me?
Interesting question. The first hint that a user is connecting to a "Telnet" socket is the port number, the official port is 23, however the server could be listening on any port.
The biggest issue with detecting Telnet is that there isn't always a strict "Telnet" protocol in use (although one exists) because the server can choose not to implement it and simply talk straight vanilla ASCII. You're essentially dealing with plain ASCII characters and (optional) control characters travelling over a plain old TCP connection. There are some big hints that most telnet servers give out that give them away though, but you'll need to inspect the traffic to be sure.
The things to look for in the network traffic are:
The traffic is mostly ASCII characters, usually with English words or linux program names in the mix.
Control characters. These are not guaranteed to be implemented by all Telnet servers (or the terminal sitting behind it), but almost always are. Specifically look for CSI control codes, which do things like change terminal size, colour, and cursor position.
The smoking gun, Telnet Negotiation. Usually the Telnet server sends these commands upon connection, and the client can respond with information as well. It handles some other functions such as terminal resizing and character sets as well, but again, not all servers implement this. From this info, you can usually scrape what client application the client is using (eg Putty).
Detecting any of these things in a TCP connection is a pretty big indication that a Telnet session is taking place.