Lately I've been experimenting with creating a multiplayer game.
In the game each client sends a UDP datagram to the server every time he moves.
Then every X seconds the server sends to all the clients the current state of the game so each client can render all of the other clients.
However when implementing the server I ran into a complication upon receiving datagrams from the clients.
If, lets say, Client A starts sending UDP datagrams to the server and then crashes for whatever reason.
If the server tries to send a message back to the client, since no port is listening on the UDP port on the client's side, his machine will respond with an ICMP packet that contains a ConnectionReset. However, when receiving such packet, I don't have any way through code to know from which remote end point this was received.
Moreover, I've read that the best way to deal with this scenario is simply to close the UDP socket on which this was received (but I obviously don't want to close it, so the server can continue running).
Code to reproduce the scenario:
IPEndPoint serverEndp = new IPEndPoint(IPAddress.Loopback, 8080);
UdpClient server = new UdpClient(serverEndp);
// New client
UdpClient client = new UdpClient();
client.Send(new byte[] { 1, 2, 3 }, 3, serverEndp);
// Simulating that the client has crashed
client.Close();
client.Dispose();
IPEndPoint clientEndPoint = null;
// Server received a datagram from a client which has crashed.
server.Receive(ref clientEndPoint);
// Server doesn't know the client has crashed and tries to send him data back.
server.Send(new byte[] { 4, 5, 6 }, 3, clientEndPoint);
try
{
// The client's machine sends the server an ICMP ConnectionReset datagram.
// When this datagram is received this will throw an exception.
server.Receive(ref clientEndPoint);
}
catch (SocketException se)
{
Console.WriteLine(SocketError.ConnectionReset == se.SocketErrorCode); // True
Console.WriteLine("How to deal with this?");
}
I have several questions:
Should I change my approach to have an extra TCP connection and then determine by that when the client has crashed?
What is the best way to deal with the above scenario?
Related
So I've been having a tough time finding documentation on exactly how sockets should behave when you have 2, both bound to the same endpoint, but one of them is also connected to a remote endpoint.
The sockets are UDP IPv4
Running in .net core 2.2/3 on linux x64
What I have been able to gather from various sources, is that the connected socket should always and only receive datagrams from the endpoint it is connected to and the "unconnected" socket will receive everything else.
I vaguely remember reading that the kernel socket implementation assigns "points" to each socket when a dgram arrives, and the socket with the higher score (most specific route) gets the data. If two socket get the same score, the dgrams are "load balanced" between the sockets.
I put together a small test:
class Program
{
static void Main(string[] args)
{
var localEp = new IPEndPoint(IPAddress.Loopback, 1114);
var remoteEp = new IPEndPoint(IPAddress.Loopback, 1115);
//Socket bound to local EP, not connected should receive from everyone
var notConnected = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
notConnected.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
notConnected.Bind(localEp);
//Socket bound and connected should receive from only it's remote EP
var connected = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
connected.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
connected.Bind(localEp);
connected.Connect(remoteEp);
var notConnectedTask = Task.Run(() => Receive(notConnected, "Not Connected"));
var connectedTask = Task.Run(() => Receive(connected, "Connected"));
//Remote socket to send to connected socket
var remote1 = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
remote1.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
remote1.Bind(remoteEp);
remote1.Connect(localEp);
//Remote socket to send to notConnected socket
var remote2 = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
remote2.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
remote2.Bind(new IPEndPoint(IPAddress.Loopback, 1116));
remote2.Connect(localEp);
for (int i = 0; i < 10; i++)
{
//This should be received by connected socket only
remote1.Send(Encoding.Default.GetBytes($"message {i} to connected socket"));
//This should be received by unconnected socket only
remote2.Send(Encoding.Default.GetBytes($"message {i} to notConnected socket"));
}
remote1.Send(Encoding.Default.GetBytes("end"));
remote2.Send(Encoding.Default.GetBytes("end"));
Task.WaitAll(notConnectedTask, connectedTask);
}
public static void Receive(Socket sock, string name)
{
EndPoint ep = new IPEndPoint(IPAddress.Any, 0);
var buf = new byte[1024];
Console.WriteLine($"{name} is listening...");
while (true)
{
var rcvd = sock.ReceiveFrom(buf, ref ep);
var msg = Encoding.Default.GetString(buf.Take(rcvd).ToArray());
Console.WriteLine($"{name} => {msg}");
if (msg.SequenceEqual("end"))
return;
}
}
}
To My surprise and chagrin, the result was nothing close to what I expected:
Connected is listening...
Not Connected is listening...
Connected => message 0 to connected socket
Connected => message 0 to notConnected socket
Connected => message 1 to connected socket
Connected => message 1 to notConnected socket
Connected => message 2 to connected socket
Connected => message 2 to notConnected socket
Connected => message 3 to connected socket
Connected => message 3 to notConnected socket
Connected => message 4 to connected socket
Connected => message 4 to notConnected socket
Connected => message 5 to connected socket
Connected => message 5 to notConnected socket
Connected => message 6 to connected socket
Connected => message 6 to notConnected socket
Connected => message 7 to connected socket
Connected => message 7 to notConnected socket
Connected => message 8 to connected socket
Connected => message 8 to notConnected socket
Connected => message 9 to connected socket
Connected => message 9 to notConnected socket
Connected => end
Not only did the notConnected socket not receive anything, but the connected socket got everything...
So neither of my expectations seem true. No load balancing, and no point system.
I had posted a comment on SO asking this question and got a reply almost confirming my expectation:
I think if one is connected to a remote endpoint, then all datagrams
originating from that remote endpoint will end up at the connected
socket. The unconnected one would only catch datagrams from other
remote endpoints.
And I also have an e-mail from the OpenSSL mailing group mostly confirming it as well...
I suppose the above test should answer my question definitively, but it just seems so wrong!
Perhaps I made a mistake in the code, or I'm just missing something. I'd appreciate a bit of guidance.
Am I completely wrong about how sockets work?
EDIT
So I just re-ran my test, exactly as above, and the result is almost the same, only the socket receiving the data is the "notConnected" socket.
Binding the notConnected socket after the connected socket also has no effect.
I vaguely remember reading that the kernel socket implementation assigns "points" to each socket when a dgram arrives, and the socket with the higher score (most specific route) gets the data. If two socket get the same score, the dgrams are "load balanced" between the sockets.
I would be curious to know where you read that. Because it's not consistent with anything I've ever read or heard about reusing socket addresses.
Rather, my understanding has always been that if you reuse a socket address, the behavior is undefined/non-deterministic. For example:
Once the second socket has successfully bound, the behavior for all sockets bound to that port is indeterminate.
When I run your test code, I get behavior opposite from which you report. In particular the "Not Connected" socket is the one that receives all of the traffic.
When I modify the code so that both sockets call Connect(), one each to each of the remote endpoint addresses, only one socket winds up getting any datagrams. This is also consistent with my understanding and the previous test. In particular, Connect() on a connectionless-protocol socket operates at the socket level, filtering out any datagrams the socket receives before the application sees them.
So, on my computer the "Not Connected" socket is the one that's getting all the traffic, and if I tell it to connect to one of the remote endpoints that is sending datagrams, then while it still receives all of the traffic, my application sees only those datagrams it asked for with the Connect() call. The other datagrams are discarded.
(As an aside: in my view, "connecting" a socket that is using a connectionless protocol should be considered simply a convenience, and should not be viewed as actually connecting the socket. The same socket can still send datagrams to other remote endpoints, via SendTo(), and the socket is still receiving traffic from other remote endpoints, your program is just not getting to see that traffic.)
For reused socket addresses, I have also seen in the past, traffic delivered randomly. I.e. sometimes one socket gets the traffic, sometimes the other one does. The fact that there is at least some socket that is consistently receiving the traffic is an improvement over that!
But nonetheless, I don't believe you should have any reason to ever expect SocketOptionName.ReuseAddress to work reliably. It's not documented to do so, and in my experience it does not. Both the results you report, as well as the different results I obtained with the same code, are entirely consistent with the "non-deterministic" nature of reusing socket addresses.
If you have seen anything that claims that reusing socket addresses can and/or should produce some deterministic result, I would say that reference is simply incorrect.
So reading over the linux man pages for SO_REUSEPORT, I came across this:
For UDP sockets, the use of this option can provide better
distribution of incoming datagrams to multiple processes (or
threads) as compared to the traditional technique of having
multiple processes compete to receive datagrams on the same
socket.
So it would seem that SO_REUSEPORT is what I need, rather than SO_REUSEADDRESS. Which is unfortunate because SO_REUSEPORT is not available on windows...
Also, to confirm Peter Duniho's answer, straight from the horse's mouth:
The motivating case for so_reuseport in UDP would be something like a
DNS server. An alternative would be to recv on the same socket from
multiple threads. As in the case of TCP, the load across these
threads tends to be disproportionate and we also see a lot of
contection on the socket lock. Note that SO_REUSEADDR already allows
multiple UDP sockets to bind to the same port, however there is no
provision to prevent hijacking and nothing to distribute packets
across all the sockets sharing the same bound port.
To sum it up, multiple UDP sockets bound to the same endpoint with SO_REUSEADDRESS set will have undefined behavior. That is to say there is no way to tell where the data will end up.
Multiple UDP sockets bound with SO_REUSEPORT will see the dgrams distributed in a sort of "load balanced" way.
As I still don't know how one connected/bound socket and one bound socket will behave with SO_REUSEPORT, I will test my scenario above with SO_REUSEPORT and update this answer.
So this commit to the linux kernel does in fact implement the "socket scoring" system I thought I had read about. Specifically, static int compute_score seems to take a "udp table" and compute the socket with the highest score for a given datagram. This should guarantee that a connected socket will receive dgrams from it's endpoint, even when another socket is also bound to the same local endpoint.
This is a gist I created to test this case. It works as I had hoped, with the connected socket always receiving dgrams from it's remote endpoint.
I've created two apps (A client and a server) which can communicate with each other as long as I input the local IP address of the machine the server app is running on into the client app (in code).
I would like the client app to automatically discover the local IP address of the machine running the server app and connect to it, so they can be run on any network without the need to enter the IP in code.
Both of these apps with be running on the same network (ie. Over WiFi, not the Internet)
Here is what I have so far in my client app:
// COMMUNICATE WITH SERVER
private TcpClient client = new TcpClient();
private IPEndPoint serverEndPoint = new IPEndPoint(IPAddress.Parse("192.168.2.35"), 8888);
public Console()
{
InitializeComponent();
client.Connect(serverEndPoint);
}
private void SendMessage(string msg)
{
NetworkStream clientStream = client.GetStream();
ASCIIEncoding encoder = new ASCIIEncoding();
byte[] buffer = encoder.GetBytes(msg);
clientStream.Write(buffer, 0, buffer.Length);
clientStream.Flush();
}
In this example I can only connect to a server running on "192.168.2.35", I would like it to be able to find the server running on port 8888 on any machine on the network.
Alternatively, if this isn't possible I would like the server to broadcast its IP as a message (of some sort) and have the client receive this message and verify it is the server and connect to it.
I think my second example is the proper way to do this, but I can't seem to wrap my head around how to get it working (I'm fairly new to C#), and any other examples I've found I can't seem to get to work with my applications.
Here is my server code if it helps answer my question:
private void Server()
{
this.tcpListener = new TcpListener(IPAddress.Any, 8888);
this.listenThread = new Thread(new ThreadStart(ListenForClients));
this.listenThread.Start();
}
private void ListenForClients()
{
this.tcpListener.Start();
while (true)
{
TcpClient client = this.tcpListener.AcceptTcpClient();
connectedClients++;
lblNumberOfConnections.Text = connectedClients.ToString();
Thread clientThread = new Thread(new ParameterizedThreadStart(HandleClientComm));
clientThread.Start(client);
}
Thanks
EDIT: I've tried adding THIS to my project, but being so new I'm unsure how to implement it properly, and it didn't get me anywhere.
2nd EDIT: I've tried implementing a UDP broadcast a few times now, with no luck on any attempt yet. My latest attempt was at implementing THIS (minus the chat parts). I just can't seem to get a UDP broadcast working at all with my project, as it seems to be way over my head at my current skill level. Unfortunately, having my client automatically connect to the server is 100% necessary for my project to function...
My other problem, which is maybe best to start a separate question for, but somewhat correlates to this issue is: My client GUI consists of a panel that switches between multiple custom classes, each containing different buttons, etc. (works similar to tab pages) that communicate to the server I'm trying to connect to. Once I get the UDP broadcast figured out, will I need to code that into every class separately? or is there a way of having all classes running in my panel connect to the same server?
A simple, but possibly costly(in terms of network traffic) solution would be for your server application to broadcast over UDP it's application and connection info. Your client could listen for all broadcast packets that have your servers custom header. Assuming a connection is made you could stop the broadcast. The downside is you would have to be broadcasting constantly if a client is not connected and this can clog your network if there aren't limits placed on the broadcast speed.
EDIT: Here is a boiled down explanation generated from the MSDN article https://msdn.microsoft.com/en-us/library/tst0kwb1(v=vs.110).aspx?cs-save-lang=1&cs-lang=csharp#code-snippet-1
EDIT #2: I've expanded on this answer on my blog, as well as provided downloadable example projects. the article can be found at http://martialdeveloper.com/wordpress/?p=21
1. Find your network's broadcast IP
A special “Broadcast Address” must be used when using UDP for the purpose of sending a datagram to all machines connected to a given network. For example, the typical home network host/gateway of 192.168.0.1 has a broadcast address of 192.168.0.255. If your network differs from this you can use an IPv4 broadcast address calculator like the one found here http://jodies.de/ipcalc
Or read the introductory section on MSDN describing the broadcast address. https://msdn.microsoft.com/en-us/library/tst0kwb1(v=vs.110).aspx?cs-save-lang=1&cs-lang=csharp#code-snippet-1
2. Select a listening/broadcast port
Any port that is free on your client & server is fine. The MSDN example uses 11000. This port number is used in your broadcaster, and listener.
3. Code for the Listener
Note to the reader. All error handling has been omitted for clarity of the example.
int listenPort = 11000;
bool done = false;
UdpClient listener = new UdpClient(listenPort);
IPEndPoint groupEP = new IPEndPoint(IPAddress.Any,listenPort);
while (!done) // This loop listens for your broadcast packets
{
Console.WriteLine("Waiting for broadcast");
byte[] bytes = listener.Receive( ref groupEP);
Console.WriteLine("Received broadcast from {0} :\n {1}\n",
groupEP.ToString(),
Encoding.ASCII.GetString(bytes,0,bytes.Length));
}
listener.Close();
Note: The third parameter to Console.WriteLine, "Encoding.ASCII..." represents the string value sent over UDP in the datagram packet. This contains the desired negotiation information for a discovery situation, such as the IP address of the client or server you wish to connect to.
4. Code for the Broadcaster
Socket s = new Socket(AddressFamily.InterNetwork, SocketType.Dgram,
ProtocolType.Udp);
IPAddress broadcast = IPAddress.Parse("This string should be the broadcast IP address"); //NOTE: Broadcast IP goes here!!
byte[] sendbuf = Encoding.ASCII.GetBytes("This is the message string to be broadcast"); //Your message to the client/server goes here, I.E. an
// app/client name or ID and an IP to connect with over TCP
IPEndPoint ep = new IPEndPoint(broadcast, 11000);
s.SendTo(sendbuf, ep);
Console.WriteLine("Message sent to the broadcast address");
NOTE: This is a very simple example. The broadcaster may need to rebroadcast for a period of time to make sure the Listener receives it. Even after the UDP datagram is sent/received there will need to be some negotiation to ensure the TCP connection is made properly.
I have the following setup:
Dedicated server --> Internet --> Modem (telenet) --> Router --> client
The client initiates a tcp connection with the server to register itself on the server, and gives through following info:
mac address of the client
external ip; this is retrieved by using webclient string download from whatsmyip.org
Some updates occur on the server and of course the client needs to be notified, so the client can start a sync session on its own:
To notify the client, the server sends a udp packet from the server to the modem (to the external ip, earlier received from the client), in the meanwhile the client is listening for udp packets behind the router.
The problem is that I'm not receiving any packets.. Is my scenario possible, what should I do?
Requirements:
Solving this by enabling port-forwarding on the router isn't an option
The server has a fixed ip
The client can be disconnected from the internet, at times
The solution has to work on different kinds of routers
Both ports at which packets are send & received are the same
All programming is done in C#
The server notifies the client when there is an update, the client may never poll the server for updates to prevent overload (in case several clients are doing this the same time)
Greets Daan & thanks in advance
EDIT:
Code example from server:
UdpClient udpSender = new UdpClient();
IPEndPoint localServerGateway = new IPEndPoint(IPAddress.Parse(externalIp), 8003);
string message = "testmessage";
byte[] messageBytes = Encoding.ASCII.GetBytes(message);
try
{
udpSender.Send(messageBytes, messageBytes.Length, localServerGateway);
}
catch (Exception error)
{
Console.WriteLine("Error while sending message: " + error.ToString());
}
udpSender.Close();
Code example from client:
private void listenForMasterSyncRequest()
{
bool done = false;
IPEndPoint groupEP = new IPEndPoint(IPAddress.Any, 8003);
try
{
while (!done)
{
byte[] bytes = masterSyncUdpListener.Receive(ref groupEP);
handleMessage(bytes, bytes.Length, true); // handles incoming messages, this is never reached because no packets are received :-(
}
}
catch (Exception e)
{
Console.WriteLine("An error occured while listening to server broadcast updates: " + e.ToString());
}
finally
{
masterSyncUdpListener.Close();
}
}
NAT works by setting up sessions between external and internal hosts. But the session must be initiated on the internal side, and in your case that's the client side. So the way it has to work is that the client has to poll the server, sending a UDP packet to a particular port on the server asking if a sync is needed. The server must send a UDP response from that same port back to the same port the client sent the original request. If you do it this way packets from the server will get through, otherwise they will not. I know this works because this is exactly how DNS lookups work from behind NAT.
Since you don't have control of the NAT devices in the path, the only sane way here is to use TCP as your main transport.
I have created a windows service socket programme to lisen on specific port and accept the client request. It works fine.
protected override void OnStart(string[] args)
{
//Lisetns only on port 8030
IPEndPoint ipEndPoint = new IPEndPoint(IPAddress.Any, 8030);
//Defines the kind of socket we want :TCP
Socket serverSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
//Bind the socket to the local end point(associate the socket to localendpoint)
serverSocket.Bind(ipEndPoint);
//listen for incoming connection attempt
// Start listening, only allow 10 connection to queue at the same time
serverSocket.Listen(10);
Socket handler = serverSocket.Accept();
}
But I need the service programme to listen on multiple port and accept the client request on any available port.
So I enhanced the application to bind to port 0(zero), so that it can accept the request on any available port.
But then I got the error 10061
No connection could be made because the target machine actively refused it.
I am unable to know whats the reason of getting this error.
Can anybody please suggest the way to enhance the code to accept the request on any port.
But the client need to send request to connect to specific port. e.g client1 should connect to port 8030, client2 should connect to port 8031.
So I enhanced the application to bind to port 0(zero), so that it can accept the request on any available port.
Wrong. 0 means that the OS should assign a port. A server can only listen at one port at a time. The listen socket just accepts new connections.
The new connection will have the same local port, but the combination of Source (ip/port) and destination (ip/port) in the IP header is used to identify the connection. That's why the same listen socket can accept multiple clients.
UDP got support for broadcasts if that's what you are looking for.
Update:
A very simplified example
Socket client1 = serverSocket.Accept(); // blocks until one connects
Socket client2 = serverSocket.Accept(); // same here
var buffer = Encoding.ASCII.GetBytes("HEllo world!");
client1.Send(buffer, 0, buffer.Count); //sending to client 1
client2.Send(buffer, 0, buffer.Count); //sending to client 2
Simply keep calling Accept for each client you want to accept. I usually use the asynchronous methods (Begin/EndXXX) to avoid blocking.
I'm having a problem with sockets.
I've written myself two simple applications, one server and one client.
The server simply waits for a UDP packet to arrive, printing out something in the console once that happens.
The client sends a UDP packet to a specific end point.
// server
Socket sock = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
sock.Bind(new IPEndPoint(IPAddress.Any, 1337));
IPEndPoint remote = new IPEndPoint(IPAddress.Any, 0);
byte[] buf = new byte[1024];
sock.ReceiveFrom(buf, ref remote);
Console.WriteLine("Received packet.");
// client
Socket sock = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
IPEndPoint remote = new IPEndPoint(IPAddress.Parse(Console.ReadLine()), UInt16.Parse(Console.ReadLine()));
byte[] buf = { 1, 2, 3, 4 }; // random data
sock.SendTo(buf, remote);
Now the weird thing is that when the client socket sends its packet to the public IP address of my router, the server socket only receives it if the client is not run on the same PC as the server. So if I start the server on my PC and then start the client on my PC, enter my public IP and port 1337, the server doesn't receive anything.
However, if I send the client application to my friend and give him my IP address and port, it works perfectly fine.
It also works if I let the client connect to my local IP instead of my public one.
Am I the only one experiencing this behaviour?
Port 1337 is forwarded to the computer the server is ran on, btw.
This seems to be a NAT configuration issue. You must have configured nat on the router to forward packets coming on the public interface with specific port (1337) to be forwarded to the server. So this works when your friend sends you a UDP packet.
But you must not (don't know if its even possible) have configured natting the other way around meaning the same UDP packet coming to the internal interface. This is the case when you send packet from server to client with both on the same machine.
When both the server and client (on a single machine or two different machines) are on your internal network it will be best to use the server's interface id than depending on natting
Most routers and modems do not normally forward UDP traffic -- see http://www.gotroot.com/blogpost4-Why-your-should-never-forward-UDP-out-of-your-firewall for an explanation. Also try attaching both the client and server to the same physical network (well, subnet) and to try sending datagrams directly to the server instead of forwarding them.