This question is acting as an aggregation of what I've learned so far about this subject. It seems to not be discussed that much (maybe because it's not used that often?).
The only other existing posts are several years old and have few solid answers, but do introduce new ideas which I believe are leading me in the right direction. So I want to lay them all out in this question. So hopefully all of those questions could be answered here for me AND for anyone else who is also asking this. Let this be a guide for anyone else.
My problem: (context)
I discovered the TCPClient/TCPListener api (System.Net.Sockets) recently and wanted to mess around with it to learn a little more about networking. Microsoft docs provide "starter code" to introduce you to its basic functionality. https://learn.microsoft.com/en-us/dotnet/api/system.net.sockets.tcpclient?view=net-6.0
https://learn.microsoft.com/en-us/dotnet/api/system.net.sockets.tcplistener?view=net-6.0
(note, I am aware of FTP)
Once I started tinkering, I decided I wanted to make a program to send a file over my LAN (my PC to my Laptop) using their private IP's. I eventually got it working.
My issues started when I then said to myself "Now let's see if I can send a file to my friend".
(note, Yes, my file was not encrypted, I only sent insensitive info like a PNG with scribbles, for testing purposes)
My code is set up so the "file receiver" is also the server host (TCPListener). I sent my friend the program and showed him how to set it up as a file receiver (host). When I tried to send the file on my end, I got an error saying:
"A connection attempt failed because the connected party did not properly respond... etc". Which was a "SocketException (10060)"
I did make sure to connect using his public IP, and we were using the same port. And I did various other things to check and see what might be causing the issue. No Fix.
I then asked my friend to try and send a file to me (I would be the server). But this time, I asked him to use port "25565" because I had forwarded that port for Minecraft a long time ago. Previously we used "13000", but we both switched to 25565. And my friend managed to send a file across to me successfully.
So it seems to me that the issue lies in port forwarding. I did not try to test if it was firewall blocking, but part of me doubts that.
My Primary Question:
For C#'s System.Net.Socket library, how can I simply achieve a "peer-to-peer" WAN connection between 2 PCs without the need for "interventions" like port forwarding?
I looked in various places to find example code maybe showing how the "Socket" class might be used in a certain way to achieve this, but I found nothing after various Google searches. What I did find were basic concepts/terminologies that claimed to be the solution.
Like: Sending TCP Packets to outside computer without Port Forwarding
https://serverfault.com/questions/604644/access-to-a-network-server-without-port-forwarding
TCP NAT-Traversal /- Punching with .NET
But I'm looking for how those concepts would be achieved with this particular library. Or if it's not possible with System.Net.Sockets, is there another C# library that is more suited to my situation?
There is a "AllowNatTraversal" option for TCPListeners, but that didn't make it work. So my state with this issue is that maybe I have to use the underlying Socket class in a very certain way to do "TCP hole punching", I'm not sure. This is the first time I've ever messed with a networking api that is this low level.
I can provide my code, but this post is already getting long. If it's needed I will amend this question with that code.
Ultimately:
This question also segways into other questions I've had about networking. Because of a larger game project I am working on which I am considering having a multiplayer option.
Most games with a multiplayer option usually require you to forward ports (going into your router's settings). But some people don't have that option, but still want to be able to play games with friends in a "privately setup lobby".
Like Terraria... I have tried to also research into how Terraria achieves its "host and play" multiplayer, and did not come up with a definitive answer. Everyone just calls it "peer-to-peer" or "hosted peer-to-peer", but no granular info is provided on how that works, or what libraries they use.
It seems by default, you can't just host a server without having to expose ports in router settings (more work for the user), which I understand is the preferred method, but doing so could also open you to vulnerabilities. And some people don't have the accessibility.
I want a "built-in" option for allowing some sort of WAN connection/communication that doesn't require port forwarding. I know it's possible, and in my current case, I'm wondering how I would do that for System.Net.Sockets.
I am aware of things like UPnP, which I've heard is not that good.
Finally:
If there is a better place to ask this question, like the networking part of StackOverflow, I can move my post there. But I am asking about a particular library in this case.
And I hope I've explained enough so any experts out there can help. And I hope this post can serve as a paved road for anyone else trying to learn about this particular form of networking. Because I believe many game developers would want to also have "player hosted peer-to-peer" multiplayer in their games. Because it doesn't require a dedicated server to run. And requires the least amount of work to set up.
This post is VERY LONG, so look at the headings to figure out what you want to read first. I've updated this post over multiple days. And added new things, some things may be repeated for each main heading.
But at this point in time, I recommend reading "Answer #2" down below. And also read "Some important info going forward".
The reason why you can't just have 2 UDP (or TCP) clients sending packets to each other over WAN (using public IP addresses) is because the devices that receive the packets, like your router, don't know which computer on the LAN (connected to the router) to send it to; it doesn't have enough information on where to send it.
Your router needs to be told, in some way, where to route packets. Achieving peer-to-peer connections requires you to use some method or technique to tell YOUR router where to route things.
"Hay router, I'm gonna send a packet to [THIS IP AND PORT]. I am expecting a response, if you get anything from that IP and port, send it to me"
"Hay router, if you get any packets with [THIS PORT], send them to me"
The following are the ways I've discovered for doing this. Personally, I prefer NAT Protocols like "NAT-PMP" and "UPnP" (Answer #2)
I did find this video which honestly helps explains things a lot on the topic.
https://www.youtube.com/watch?v=TiMeoQt3K4g
And a sequel where he made a python implementation
https://www.youtube.com/watch?v=IbzGL_tjmv4
From this, I managed to make a C# implementation of NAT Hole Punching for UDP. I assume this same technique could be applied to TCP as well? If anyone could comment on that, or if the technique has to be modified to work for tcp.
// This code could be in Main, or another static function
Console.WriteLine("enter the other persons IP:");
string destinationIP = Console.ReadLine();
Console.WriteLine("type a message you want to send:");
string inputMessage = Console.ReadLine();
int sourcePort = 25000;
int destinationPort = 25001;
// Send a "dummy packet" which "punches a hole" in your router, so each router knows where to route the packets.
UdpClient sender = new UdpClient();
sender.Client.Bind( new IPEndPoint(IPAddress.Parse("0.0.0.0"), sourcePort) );
sender.Client.SendTo(new byte[] { 2 , 4 , 6 , 8 }, new IPEndPoint(IPAddress.Parse(destinationIP), destinationPort) );
sender.Close();
Console.WriteLine("Punched Hole...");
// Start a udp client to listen for incoming packets
Thread listenerThread = new Thread(() => {
UdpClient listener = new UdpClient();
listener.Client.Bind(new IPEndPoint(IPAddress.Parse("0.0.0.0"), sourcePort));
byte[] buffer = new byte[1024];
Console.WriteLine("Server Listening:");
while(true) {
int bufferSize = listener.Client.Receive(buffer);
Console.WriteLine("Recieved Message:");
string message = Encoding.UTF8.GetString(buffer, 0, bufferSize);
Console.WriteLine(message);
Console.WriteLine();
}
});
listenerThread.Start();
// waiting a bit of time just in case
Thread.Sleep(5000);
// Start a different client that will be sending actual data between the clients, through the punched holes
UdpClient sender2 = new UdpClient();
sender2.Client.Bind(new IPEndPoint(IPAddress.Parse("0.0.0.0"), destinationPort));
byte[] sendData = Encoding.UTF8.GetBytes(inputMessage);
Console.WriteLine("Preparing to send data...");
for(int i = 0; i < 10; i++) {
Thread.Sleep(1000);
sender2.Client.SendTo(sendData, new IPEndPoint(IPAddress.Parse(destinationIP), sourcePort));
}
I have a feeling there is a more elegant way to do this, but this does indeed work. I was able to test this code with a friend (because I don't want to spend money to do a simple test on a cloud server).
This code has to run on both computers (yours and the other peer) at relatively the same time. Not at exactly the same time though, but within a span of seconds. Because "hole punching" has a very short "expiration date". So they have to happen close to each other in time. You can coordinate this over Discord or another chat program.
You could also make the program send out "dummy packets" once every few seconds until a connection is made. So you wouldn't have to time things necessarily
On that note, even after a hole punch has succeeded, if no data is being sent back and forth for many seconds (it probably varies between routers), then the "hole" that was punched will be "patched up", and you would have to restart the process again. So you may need to also send dummy packets once every few seconds to keep the connection alive. Or you can be smart about it and only send a dummy packet if no other 'actual packets' were sent within the last few seconds.
"NAT" is a thing that most routers (I would imagine) are capable of today. NAT is responsible for figuring out which PC to send an incoming packet to. But sometimes your router needs a little bit of input from its connected PCs to know what to do.
This fact also applies to hole punching, but ideally, you might not want to send "dummy packets" just to "punch a hole" and then send "keep-alive packets" every 10 seconds to keep the connection open. It's a waste of network resources.
Ideally, you'd want a PC to talk directly to the router to set up a "temporary port forward" that might last for a couple of hours, and you can "renew the port mapping" once an hour by sending a message to the router; so no outbound WAN packets are sent.
This is what "NAT Protocols" allow you to do; Portforwarding directly from your application.
I'm glad you asked!
https://en.wikipedia.org/wiki/Port_Control_Protocol#PCP_as_a_solution
This Wikipedia page lists quite a few libraries for different programming languages, at the bottom. But for C# I have chosen to use a NuGet package called "Mono.Nat" which supports "NAT-PMP" and "UPnP". From what I've heard, the "NAT-PMP" protocol is the preferred one, but either is probably good.
Here is their Github which has links to their NuGet as well. But you can just search up "Mono.Nat" in the NuGet Package Manager of your IDE (VS or Rider).
https://github.com/alanmcgovern/Mono.Nat
Also, here is some example code I used for testing NAT-PMP, and actually performing the software port-forward. This code essentially sends a message (packet) to your router saying: "Hay, can you forward incoming packets with [THIS PORT] to me?". It may not always succeed depending on your scenario. And this code does not properly handle those errors completely. But it will tell you if there was some sort of error. So this code should be improved for production uses.
public static bool SetupPortForward() {
int portToForward = 25000;
bool portMapSet = false;
bool fail = false;
NatUtility.DeviceFound += async (object sender, DeviceEventArgs args) => {
try {
INatDevice device = args.Device;
// Only interact with one device at a time. Some devices support both
// upnp and nat-pmp.
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine("Device found: {0}", device.NatProtocol);
Console.ResetColor();
Console.WriteLine("Type: {0}", device.GetType().Name);
Console.WriteLine("IP: {0}", await device.GetExternalIPAsync());
Console.WriteLine("---");
await device.CreatePortMapAsync(new Mapping(Protocol.Udp, portToForward, portToForward));
//await device.DeletePortMapAsync(new Mono.Nat.Mapping(Mono.Nat.Protocol.Udp, 25000, 25000));
//Mono.Nat.Mapping[] mappings = await device.GetAllMappingsAsync();
//foreach(var mapping in mappings) {
// Console.WriteLine(mapping);
//}
portMapSet = true;
} catch(Exception e) {
Console.WriteLine(e.Message);
fail = true;
}
};
// Replace "192.168.0.1" with your default gateway, or use a function which finds your default gateway automatically
string defaultGateway = "192.168.0.1";
NatUtility.Search(IPAddress.Parse(defaultGateway), NatProtocol.Pmp);
while(true) {
if(portMapSet || fail) break;
Thread.Sleep(1);
}
return !fail;
}
Note: This code notably needs namespaces "Mono.Nat" and "System.Net". So you will also need the NuGet package "System.Net.Sockets" so you can use the "IPAddress" class.
But if you are using a different C# library for networking (like SFML maybe?) You could alternatively use the NuGet package "System.Net.Primitives" which just provides the IPAddress class.
I should also note that my example code is based off of code inside the Mono.Nat Github repository; At "Mono.Nat.Console/Main.cs". So there may be some more useful info in there that I missed.
https://github.com/alanmcgovern/Mono.Nat/blob/master/Mono.Nat.Console/Main.cs
But the example code I show above SHOULD automatically setup a port forward for port 25000, but you can change that. The code also assumes everyone's default gateway is the same "192.168.0.1". But it may be different for you, so make sure to check that using "ipconfig" in a command prompt. That ip is not my gateway, I got it from this youtube video.
https://www.youtube.com/watch?v=pCcJFdYNamc
Lastly, the port forward that is setup is TEMPORARY, so it will EXPIRE after a period of time. You can change this period of time in code, but the default (I think) is 2 hours, which is decent. Also, you can rerun this code to "renew" the expiration date, and keep it going effectively for the life of the program. The program doesn't need to make sure to "close" the port forward on exit. So if the program crashes, the port forward will "expire" and close itself on its own.
The commented out code "device.GetAllMappingsAsync()" can list off any PMP or UPnP mappings that were made, but for some reason, it only works when the "NatUtility.Search" function is using the UPnP option.
Alternatively, you can see if your router's HTML settings menu has a section for displaying logs and if you can view all existing Port Forwards.
To do this, you need to get your router's default gateway (using ipconfig) and enter the default gateway into your browser. It will take you to a page generated by your router. This page will look different for each router brand and model.
For my ASUS router, I clicked on "System Log", and then I clicked on a tab called "Port Forwarding", which lists info about existing port forwards. It shows manually made ones, and ones that were made in software (PMP). Note, it may be called "port mapping" instead.
You can keep that page open, run the example code I showed above ^, and if it succeeds. You will see a new entry in the port forward logs. Make sure to refresh if there is a refresh button on the screen. And it will show your PC's private IP. That's how the router knows where to forward the packets. And it will forward packets no matter what the external IP was.
Without any existing port forwards, if you receive packets from another public IP, you're not going to get them, obviously.
So if you have a program that is actively listening for UDP packets, but no port forwarding for the port your using, nothing is going to happen. I know, I sound like a broken record!!!
But my point is, the best way you could probably test it, is to verify that you can receive packets from an external (public) IP in the first place; from another PC out there on the interwebs.
If you're using System.Net.Sockets, this would involve creating a new UdpClient, binding it to the port you forwarded (bind ip would be 0.0.0.0), and then using one of the "receive" functions to listen for an incoming packet.
The tricky part is finding a computer with a different public IP, and running a program that sends a UDP packet. And see if the packet gets received on your PC. If it does, congratulations, it worked!
For me, the simplest thing would be to find a friend (Discord maybe?) who is willing to help. You'd have to send them a program that would send the UDP packet to your PC. You could use another existing program like Netcat or NCat (Windows version) to send the test packet.
https://sectools.org/tool/netcat/
You can then run your program (which listens for packets), but don't run the port forwarding code yet. Then, your friend will run the program which sends the packet to your pc. Nothing should happen (obviously).
But then, you re-run your program, but include the port forwarding part before listening for packets. When your program is ready, tell your friend to re-run their program, and you should receive their packet. IT'S MAGIC!!!
But if nothing happens, then the port forward probably didn't work. And you may need to make some tweaks or change something to make it work.
But ultimately, for some people in the world. This might not at all be possible. Some people live in apartments, or in motels, or are on vacation in a hotel, using the hotel's WiFi. So you may be unfortunately restricted from making temporary port forwards. For people who have decent "access" to their home's router. This should be possible though. But this method does help a lot more people to be able to host [game] servers on their PC without needing to have direct access to their router's settings. Unfortunately, this won't work for absolutely everyone. And hole punching will become the one fallback method for some of you.
Lastly, I want to show some example code that you can use to test the port forward.
This code is for the server (the packet receiver, aka you).
int sourcePort = 25000; // should be the same as your port forward
UdpClient udpServer = new UdpClient();
IPEndPoint localEndPoint = new IPEndPoint(IPAddress.Parse("0.0.0.0", sourcePort);
udpServer.Client.Bind(localEndPoint);
byte[] buffer = new byte[1024];
EndPoint incomingEndPoint = localEndPoint;
int bufferSize = udpServer.Client.ReceiveFrom(buffer, ref incomingEndPoint); // incomingEndPoint will contain the public ip of the sender
Console.WriteLine($"Incoming Packet From: {incomingEndPoint}");
string message = Encoding.UTF8.GetString(buffer, 0, bufferSize);
Console.WriteLine($"Message: {message}");
Console.ReadLine();
And the following is the code that would run on your friend's PC (or another remote PC).
// You probably don't need to use this port for the sender
// The sender can use any port they want, and their router will use NAT hole punching to figure things out.
// But both the client and server could forward this port if they want. But it's not needed. Only the server needs to forward its ports.
int sourcePort = 25000;
UdpClient udpClient = new UdpClient();
IPEndPoint localEndPoint = new IPEndPoint(IPAddress.Parse("0.0.0.0", sourcePort);
udpServer.Client.Bind(localEndPoint);
byte[] sendData = Encoding.UTF8.GetBytes("Hello! This is the UDP client on the other end! If you see this, it worked!!!");
Console.WriteLine("Please enter the Public IP you wish to send the packet to, and hit enter:");
string destinationIP = Console.ReadLine();
// Here, sourcePort is refering to the server's port. This does need to be the same as what's forwarded.
// So this port could be different than the port that was used in the "Bind" function, and "localEndPoint".
IPEndPoint destinationEndPoint = new IPEndPoint(IPAddress.Parse(destinationIP), sourcePort);
udpClient.Client.SendTo(sendData, destinationEndPoint); // send it off!
Console.WriteLine("sent!");
Console.ReadLine();
You can copy/paste these lines into your editor but make it so only one or the other will run. Depending on if you press "S" or "R". For either "Send" or "Receive". So you can just have 1 program instead of 2. Your friend will have to enter your public IP. You can find your public IP by googling "my ip" or something like that. When you receive their packet, it will also print their ip. As a confirmation of where the packet came from (it could come from any ip). And this is normal, your computer also needs to know the sender's ip so it can send a return message if it wanted to.
But that is it.
This example code should give you a solid idea of how to test and make sure your "software-based port forward" actually worked. And it gives you an idea of what you may need to set up in order for the code to even work in the first place. Because sometimes the issue lies in misunderstanding the API and not doing the Bind correctly, or using the wrong ports in the wrong places. So this way is guaranteed to work. But there are other ways to use the UDPClient class.
Also, I have not tested any of this for TCP, but I suspect (and hope) it will work. You just have to change the port forward code to be for TCP.
I know this is a lot to read, but it details all the stuff I've learned about this subject. And I want to put it all in one place and organize it as best as possible. So this can serve as a good resource for other [game] programmers who want this to be an option in their application.
I have done many searches trying to find A SINGLE article/post that explains THE THING that is needed. Many of them talk about stuff that leads you in the right direction. But I feel like they never said enough.
I think many game developers would want a very simple multiplayer option in their game, that requires no pre-configuration, so people can play and enjoy their games with their friends. And I feel like this info isn't super well-known or shared that much, considering how many games just resort to using a "dedicated server" approach. Which requires manual port forward and direct access to your router's settings; not very accessible... But rants aside!
This post I think will help people who are working in C# at least. And if you're coding in a different language, then take all the stuff I talked about and find the C++, C, Java, Python, etc equivalent of it. Because they do exist. The concepts that I've talked about are "language agnostic", meaning they can apply to any language. You just need a library that allows you to interface with these "systems", like NAT-PMP. (You can now google "NAT PMP C++", and find exactly what you need)
Lastly, I would also want to extend this list with other options/methods that may be preferred in some cases. There may be more modern technologies than NAT-PMP, but for now, NAT-PMP works perfectly for me.