At the moment I'm working on a project where I have to build an RTP client and server with the .NET via C#. The current question concern the the client side.
I created a UDP Socket like the following for listening to the server :
Socket RTPSocket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
I also created an array of bytes to store received bytes :
byte[] RTPBuffer = new byte[1500];
The size is at 1500 because after packet analysis with wireshark i saw that the max packet size sent by VLC was 1428. After that i created a thread for the listening process, this thread contains the following instructions :
while(!Disposing)
{
RTPSocket.Receive(RTPBuffer);
OnRTPDataReceived(RTPBuffer);
}
The OnRTPDataReceived instruction raise an event wich is handled by the following method :
public void RTPPacketReceived(object sender, byte[] packet)
{
PacketQueue.Enqueue(packet);
}
Finally the bytes array contained in the PacketQueue Queue are processed by another thread but this is off topic.
The problem is that my socket is sometimes receiving one packet out of two and sometines one out of three and some other times every packets. In fact it's random.
I looked to this kind of problem in different forums but I didn't find any solutions to resolve my problem.
I also tried to set a better priority to the listening thread and a lower one to the others, didn't worked. Last thing I tried was reduce the number of instruction after receiving data to see if the process duration was the cause of the packet loss but this too didn't worked.
Thanks in advance for your time !
Related
I have searched for 2 days and found many, many questions/answers to what appears to be this same issue, with some differences, however none really seem to provide a solution.
I am implementing a library for controlling a DMX system (ColorKinetics devices) directly without an OEM controller. This involves communicating with an Ethernet-enabled power supply (PDS) connected to my home LAN, through a router, which drives the lighting fixtures. The PDS operates on a specific port (6038) and responds to properly formatted datagrams broadcast over the network.
I can successfully broadcast a simple DMX message (Header + DMX data), which gets picked up by the PDS and applied to connected lighting fixtures, so one-way communication is not an issue.
My issue is that I am now trying to implement a device discovery function to detect the PDS(s) and attached lights on the LAN, and I am not able to receive datagrams which are (absolutely) being sent back from the PDS. I can successfully transmit a datagram which instructs the devices to reply, and I can see the reply coming back in WireShark, but my application does not detect the reply.
I also tried running a simple listener app on another machine, which could detect the initial broadcast, but could not hear the return datagram either, however I figure this wouldn't work since the return packet is addressed to the original sender IP address.
I initially tried implementing via UdpClient, then via Sockets, and both produce the same result no matter what options and parameters I seem to specify.
Here is my current, very simple code to test functionality, currently using Sockets.
byte[] datagram = new CkPacket_DiscoverPDSRequestHeader().ToPacket();
Socket sender = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
IPEndPoint ep = new IPEndPoint(IPAddress.Parse("192.168.1.149"), 6039);
public Start()
{
// Start listener
new Thread(() =>
{
Receive();
}).Start();
sender.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
sender.EnableBroadcast = true;
// Bind the sender to known local IP and port 6039
sender.Bind(ep);
}
public void Send()
{
// Broadcast the datagram to port 6038
sender.SendTo(datagram, new IPEndPoint(IPAddress.Broadcast, 6038));
}
public void Receive()
{
Socket receiver = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
receiver.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
receiver.EnableBroadcast = true;
// Bind the receiver to known local IP and port 6039 (same as sender)
IPEndPoint EndPt = new IPEndPoint(IPAddress.Parse("192.168.1.149"),6039);
receiver.Bind(EndPt);
// Listen
while (true)
{
byte[] receivedData = new byte[256];
// Get the data
int rec = receiver.Receive(receivedData);
// Write to console the number of bytes received
Console.WriteLine($"Received {rec} bytes");
}
}
The sender and receiver are bound to an IPEndPoint with the local IP and port 6039. I did this because I could see that each time I initialized a new UdpClient, the system would dynamically assign an outgoing port, which the PDS would send data back to. Doing it this way, I can say that the listener is definitely listening on the port which should receive the PDS response (6039). I believe that since I have the option ReuseAddress set to true, this shouldn't be a problem (no exceptions thrown).
Start() creates a new thread to contain the listener, and initializes options on the sending client.
Send() successfully broadcasts the 16-byte datagram which is received by the PDS on port 6038, and generates a reply to port 6039 (Seen in WireShark)
Receive() does not receive the datagram. If I bind the listener to port 6038, it will receive the original 16-byte datagram broadcast.
Here is the WireShark data:
Wireshark
I have looked at using a library like SharpPCap, as many answers have suggested, but there appear to be some compatibility issues in the latest release that I am not smart enough to circumvent, which prevent the basic examples from functioning properly on my system. It also seems like this sort of basic functionality shouldn't require that type of external dependency. I've also seen many other questions/answers where the issue was similar, but it was solved by setting this-or-that parameter for the Socket or UdpClient, of which I have tried every combination to no avail.
I have also enabled access permissions through windows firewall, allowed port usage, and even completely disabled the firewall, to no success. I don't believe the issue would be with my router, since messages are getting to Wireshark.
UPDATE 1
Per suggestions, I believe I put the listener Socket in promiscuous mode as follows:
Socket receiver = new Socket(AddressFamily.InterNetwork, SocketType.Raw, ProtocolType.IP);
receiver.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.HeaderIncluded, true);
receiver.EnableBroadcast = true;
IPEndPoint EndPt = new IPEndPoint(IPAddress.Parse("192.168.1.149"), 0);
receiver.Bind(EndPt);
receiver.IOControl(IOControlCode.ReceiveAll, new byte[] { 1, 0, 0, 0 }, null);
This resulted in the listener receiving all sorts of network traffic, including the outbound requests, but still no incoming reply.
UPDATE 2
As Viet suggested, there is some sort of addressing problem in the request datagram, which is formatted as such:
public class CkPacket_DiscoverPDSRequest : BytePacket
{
public uint magic = 0x0401dc4a;
public ushort version = 0x0100;
public ushort type = 0x0100;
public uint sequence = 0x00000000;
public uint command = 0xffffffff;
}
If I change the command field to my broadcast address 192.168.1.149' or192.168.255.255`, my listener begins detecting the return packets. I admittedly do not know what this field is supposed to represent, and my original guess was to just put in a broadcast address since the point of the datagram is to discover all devices on the network. This is obviously not the case, though I am still not sure the exact point of it.
Either way, thank you for the help, this is progress.
So in actuality it turns out that my issue was with the formatting of the outgoing datagram. The command field needs to be an address on the local subnet 192.168.xxx.xxx, and not 255.255.255.255... for whatever reason this was causing the packet to be filtered somewhere before getting to my application, though WireShark could still see it. This may be common sense in this type of work but being relatively ignorant as to network programming as well as the specifics of this interface it wasn't something I had considered.
Making the change allows a simple UdpClient send/receive to function perfectly.
Much thanks to Viet Hoang for helping me find this!
As you've already noted, you don't need to bind to send out a broadcast but it uses a random source port.
If you adjust your code to not bind the sender, your listener should behave as expected again:
Socket sender = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
sender.EnableBroadcast = true;
Thread read_thread;
public Start()
{
// Start listener
read_thread = new Thread(Receive);
read_thread.Start();
}
The issue you've bumped into is that the operating system kernel is only delivering packets up to one socket binder (first come first serve basis).
If you want true parallel read access, you'll need to look into sniffing example such as: https://stackoverflow.com/a/12437794/8408335.
Since you are only looking to source the broadcast from the same ip/port, you simply need to let the receiver bind first.
If you add in a short sleep after kicking off the receive thread, and before binding the sender, you will be able to see the expected results.
public Start()
{
// Start listener
new Thread(() =>
{
Receive();
}).Start();
Thread.Sleep(100);
sender.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
sender.EnableBroadcast = true;
// Bind the sender to known local IP and port 6039
sender.Bind(ep);
}
Extra note: You can quickly test your udp sockets from a linux box using netcat:
# echo "hello" | nc -q -1 -u 192.168.1.149 6039 -
- Edit -
Problem Part #2
The source address of "255.255.255.255" is invalid.
Did a quick test with two identical packets altering the source ip:
https://i.stack.imgur.com/BvWIa.jpg
Only the one with the valid source IP was printed to the console.
Received 26 bytes
Received 26 bytes
I was successfully using this code on my home computer - please note that I have minified the code in order to only show the important parts
Socket Sock = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
// Connect to the server
Sock.Connect(Ip, Port);
// Setup buffers
byte[] bufferSend = ...; // some data prepared before
byte[] bufferRec = new byte[8024];
// Send the command
Sock.Send(bufferSend, SocketFlags.None);
// UNTIL HERE EVERYTHING WORKS FINE
// Receive the answer
int ct = 0;
StringBuilder response = new StringBuilder();
while ((ct = Sock.Receive(bufferRec)) > 0)
{
response.Append(Encoding.Default.GetString(bufferRec, 0, ct));
}
// Print the result
Console.WriteLine("This is the result:\n" + response.ToString());
In another environment (Windows but behind an Ubuntu firewall) I have problems receiving packets with over 1472 bytes: An exception is thrown that the request timed out.
So basically I have two options:
Either fixing the Ubuntu firewall server (how?)
Adjusting my code (probably better option?)
How would I need to adapt my code in order to split packets in working size? I thought adjusting the variable bufferRec = new byte[1024] would suffice, but obviously this does not work. I get an exception then that the received package is greater than the bufferRec. Do I have to change the SocketType?
Unfortunately I do not know a lot about sockets, so your explainations would help a lot!
Your packet is traveling various networks. Each network probably has its own MTU size. This is the maximum size a single packet can be.
If your data exceeds that size, the DF flag is checked (DF stands for Don't Fragment). If that flag is set, the packet is dropped and an ICMP response is generated.
In C# sockets, this option is controlled by the DontFragment property, wich defaults to True, hence your problem.
Note that UDP with fragmentation enabled is to be considered unreilable, since packets will probably get lost on a busy network
I’m writing some code in .Net/C# to communicate with some factory equipment over Ethernet. I first assign a handler:
_TCPConn.BeginReceive(
_StateObject.sBuffer, 0, _StateObject.sBuffer.Length, SocketFlags.None,
new AsyncCallback(Socket_DataArrival), _StateObject);
And then in my handler, first thing, I do an EndReceive, then I transfer the data from the socket’s buffer to my own input buffer for further processing later, I log some stuff, then I do a BeginReceive to start things up again and I exit the handler. So ...
....
StateObject stateObject = (StateObject)ar.AsyncState;
int bytesReceived = stateObject.sSocket.EndReceive(ar);
// transfer bytes, log stuff, then reenable receive and leave . . .
_TCPConn.BeginReceive( _StateObject.sBuffer, 0,
_StateObject.sBuffer.Length, SocketFlags.None,
new AsyncCallback(Socket_DataArrival), _StateObject);
return;
This works fine most of the time but if the factory equipment sends 2 packets close together it fails. Here's a Wireshark (network sniffer) output. The first column is the time delta from the previous packet in microseconds (yes, "micro", not "milli")
002397 10.1.1.116 10.1.2.11 TCP .... Len=6
000024 10.1.1.116 10.1.2.11 TCP .... Len=9
When this happens the first packet seems to disappear - the handler never gets called for the 6 byte packet, only the 9 byte one. I know the packet's arriving on the PC because Wireshark shows it.
BUT if we introduce a delay in the factory equipment, so instead of 24 microsec's it's 10 millisec's the problem goes away. Unfortunately that's not a solution because there's a large base of installed equipment in factories around the world we can't change.
Any suggestions for how to debug and fix this? Thank you in advance!
why are you doing EndReceive(ar) you should keep receiving. Process the received packets separately. you need not stop receiving.
I'm trying to program a simple data collection gui. The setup is as follows:
On one end, I have a programmed FPGA that's sending out simple UDP packets filled with data. This portion of the system is working - it's been verified, and I can see the packets coming in as expected in wireshark.
On the other, I'm trying to build a simple receiver in C# to collect the packets and display the data. I've tried everything I can find on UDP communications, however, and I can't seem to get the GUI to actually see any of the packets.
The packets are being sent from 192.168.0.99:1024 to 192.168.0.100:1024.
My test code is as follows:
private void ConnectToUDP(UDPOptions Options)
{
UdpClient listener = new UdpClient(1024);
IPEndPoint groupEP = new IPEndPoint(IPAddress.Any, 1024);
while (true)
{
byte[] bytes = listener.Receive(ref groupEP);
}
}
However, I can't seem to get it to do anything but block on the final line. (I.e. it blocks, but never seems to actually receive any of the data.)
Again, I can see the packets coming in on wireshark, I have my IP address on my machine set to 192.168.0.100 to actually acknowledge the incoming packets, but my program just won't see them.
Anyone have any idea what I'm doing wrong?
Thanks,
Ian
Dont know if you still need this or even if this would work for you but #INCyr, try changing the IPEndPoint port from 1024 to 0, like so:
IPEndPoint groupEP = new IPEndPoint(IPAddress.Any, 0);
This worked for me.
I'm currently in the process of developing a C# Socket server that can accept multiple connections from multiple client computers. The objective of the server is to allow clients to "subscribe" and "un-subscribe" from server events.
So far I've taken a jolly good look over here: http://msdn.microsoft.com/en-us/library/5w7b7x5f(v=VS.100).aspx and http://msdn.microsoft.com/en-us/library/fx6588te.aspx for ideas.
All the messages I send are encrypted, so I take the string message that I wish to send, convert it into a byte[] array and then encrypt the data before pre-pending the message length to the data and sending it out over the connection.
One thing that strikes me as an issue is this: on the receiving end it seems possible that Socket.EndReceive() (or the associated callback) could return when only half of the message has been received. Is there an easy way to ensure each message is received "complete" and only one message at a time?
EDIT: For example, I take it .NET / Windows sockets does not "wrap" the messages to ensure that a single message sent with Socket.Send() is received in one Socket.Receive() call? Or does it?
My implementation so far:
private void StartListening()
{
IPHostEntry ipHostInfo = Dns.GetHostEntry(Dns.GetHostName());
IPEndPoint localEP = new IPEndPoint(ipHostInfo.AddressList[0], Constants.PortNumber);
Socket listener = new Socket(localEP.Address.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
listener.Bind(localEP);
listener.Listen(10);
while (true)
{
// Reset the event.
this.listenAllDone.Reset();
// Begin waiting for a connection
listener.BeginAccept(new AsyncCallback(this.AcceptCallback), listener);
// Wait for the event.
this.listenAllDone.WaitOne();
}
}
private void AcceptCallback(IAsyncResult ar)
{
// Get the socket that handles the client request.
Socket listener = (Socket) ar.AsyncState;
Socket handler = listener.EndAccept(ar);
// Signal the main thread to continue.
this.listenAllDone.Set();
// Accept the incoming connection and save a reference to the new Socket in the client data.
CClient client = new CClient();
client.Socket = handler;
lock (this.clientList)
{
this.clientList.Add(client);
}
while (true)
{
this.readAllDone.Reset();
// Begin waiting on data from the client.
handler.BeginReceive(client.DataBuffer, 0, client.DataBuffer.Length, 0, new AsyncCallback(this.ReadCallback), client);
this.readAllDone.WaitOne();
}
}
private void ReadCallback(IAsyncResult asyn)
{
CClient theClient = (CClient)asyn.AsyncState;
// End the receive and get the number of bytes read.
int iRx = theClient.Socket.EndReceive(asyn);
if (iRx != 0)
{
// Data was read from the socket.
// So save the data
byte[] recievedMsg = new byte[iRx];
Array.Copy(theClient.DataBuffer, recievedMsg, iRx);
this.readAllDone.Set();
// Decode the message recieved and act accordingly.
theClient.DecodeAndProcessMessage(recievedMsg);
// Go back to waiting for data.
this.WaitForData(theClient);
}
}
Yes, it is possible you'll have only part of message per one receiving, also it can be even worse during transfer only part of message will be sent. Usually you can see that during bad network conditions or under heavy network load.
To be clear on network level TCP guaranteed to transfer your data in specified order but it not guaranteed that portions of data will be same as you sent. There are many reasons for that software (take a look to Nagle's algorithm for example), hardware (different routers in trace), OS implementation, so in general you should never assume what part of data already transferred or received.
Sorry for long introduction, below some advices:
Try to use relatevely "new" API for high-performance socket server, here samples Networking Samples for .NET v4.0
Do not assume you always send full packet. Socket.EndSend() returns number of bytes actually scheduled to send, it can be even 1-2 bytes under heavy network load. So you have to implement resend rest part of buffer when it required.
There is warning on MSDN:
There is no guarantee that the data
you send will appear on the network
immediately. To increase network
efficiency, the underlying system may
delay transmission until a significant
amount of outgoing data is collected.
A successful completion of the
BeginSend method means that the
underlying system has had room to
buffer your data for a network send.
Do not assume you always receive full packet. Join received data in some kind of buffer and analyze it when it have enough data.
Usually, for binary protocols, I add field to indicate how much data incoming, field with message type (or you can use fixed length per message type (generally not good, e.g. versioning problem)), version field (where applicable) and add CRC-field to end of message.
It not really required to read, a bit old and applies directly to Winsock but maybe worth to study: Winsock Programmer's FAQ
Take a look to ProtocolBuffers, it worth to learn: http://code.google.com/p/protobuf-csharp-port/, http://code.google.com/p/protobuf-net/
Hope it helps.
P.S. Sadly sample on MSDN you refer in question effectively ruin async paradigm as stated in other answers.
Your code is very wrong. Doing loops like that defeats the purpose of asynchronous programming. Async IO is used to not block the thread but let them continue doing other work. By looping like that, you are blocking the thread.
void StartListening()
{
_listener.BeginAccept(OnAccept, null);
}
void OnAccept(IAsyncResult res)
{
var clientSocket = listener.EndAccept(res);
//begin accepting again
_listener.BeginAccept(OnAccept, null);
clientSocket.BeginReceive(xxxxxx, OnRead, clientSocket);
}
void OnReceive(IAsyncResult res)
{
var socket = (Socket)res.Asyncstate;
var bytesRead = socket.EndReceive(res);
socket.BeginReceive(xxxxx, OnReceive, socket);
//handle buffer here.
}
Note that I've removed all error handling to make the code cleaner. That code do not block any thread and is therefore much more effecient. I would break the code up in two classes: the server handling code and the client handling code. It makes it easier to maintain and extend.
Next thing to understand is that TCP is a stream protocol. It do not guarentee that a message arrives in one Receive. Therefore you must know either how large a message is or when it ends.
The first solution is to prefix each message with an header which you parse first and then continue reading until you get the complete body/message.
The second solution is to put some control character at the end of each message and continue reading until the control character is read. Keep in mind that you should encode that character if it can exist in the actual message.
You need to send fixed length messages or include in the header the length of the message. Try to have something that allows you to clearly identify the start of a packet.
I found 2 very useful links:
http://vadmyst.blogspot.com/2008/03/part-2-how-to-transfer-fixed-sized-data.html
C# Async TCP sockets: Handling buffer size and huge transfers