We Keep Coding

c#, socket cannot receive fragmented UDP packets

I have an embedded device which is connected to my PC directly through ethernet.
The device is streaming audio to pc through UDP and it sends 4096 bytes UDP packet.
Given that, the MTU for ethernet is 1500 bytes the packet will be fragmented.
At PC I have a c# program that tries to receive packets and decode it. UDP receiver can receive packets very well when they have under 1500 bytes payload but it cannot receive fragmented packets.
I've monitored incoming packet by Wireshark and I can see that there is not any failure in packets nor discarded.
I don't know the problem is in my code or C# socket is Unable to receive these kinds of packets. in both cases what would be the solution?
1st Attempt: (usinfg Socket)
Socket sokcet = new Socket(SocketType.Dgram, ProtocolType.Udp);
IPEndPoint ep = new IPEndPoint(IPAddress.Any,5222);
sokcet.Bind(ep);
int counter = 0;
while (true)
{
if(sokcet.Available > 0){
byte[] bytes = new byte[sokcet.Available];
int receivedBytes = sokcet.Receive(bytes);
string print = String.Format("Packet Received : {0},{1}", receivedBytes, counter);
Console.WriteLine(print);
}
}
2nd Attempt: (using UDPClient)
IPEndPoint groupEP = new IPEndPoint(IPAddress.Any, listenPort);
UdpClient listener = new UdpClient(listenPort,AddressFamily.InterNetwork);
while (!done)
{
byte[] bytes = listener.Receive(ref groupEP);
}
None of them are working for packets larger than 1500 bytes.
Update 1:
I tested the scenario in Loopback(127.0.0.1) and I can receive 4k UDP message very well.
Update 2:
#Evk I tested the scenario from another pc connected to mine over a Switch and also over a router. now I am sure that C# does not have any problem. not sure about OS (win7 ultimate 64x).
My embedded device uses LWIP and there are some reports of a similar situation that happened when users used LWIP for sending large UDP packets. but I'm not sure this is my case.
I even checked UDP packet's Source and Destination address, Checksum and ... and I can't figure out why OS dropping my packets. is there any tools that can analyze network packet to tell if they have any problem?

I would make certain that the messages are in fact, being delivered to your processes' UDP receive queue. You may see it wireshark, but that is not a guarantee that it did not later get dropped or filtered at the UDP protocol level.
I would open a command window (CMD) and run this in a loop as you send your traffic and run your decoding application (Also, stop other applications that may use UDP as they may interfere with these figures):
> netstat -s -p udp
UDP Statistics for IPv4
Datagrams Received = 48775 <======= [This *should* increase]
No Ports = 83823
Receive Errors = 16367 <====== [ WATCH THIS]
Datagrams Sent = 130194
If you notice receive errors going up while your application sits there and does not seem to processing...then there is chance that the > 1500 byte packets are getting dropped by UDP for some reason. You may need to either increase your application's socket receive buffers or look for windows UDP network configuration options that would cause > MTU packets to be dropped on receive..

Send several requests via a socket connection

Is it possible to connect to a socket, first send a request and receive a response then send another request and receive its response within the same socket connection ?
I want to eliminate the costs of creating new socket connection for each request.

A TCP connection is an infinite bidirectional stream of bytes. There are no messages/requests at the TCP level. You can send and receive as many bytes as you want. This is even required by many application protocols.

Does TcpClient write method guarantees the data are delivered to server?

I have a separate thread on both client and server that are reading/writing data to/from a socket.
I am using synchronous TcpClient im (as suggested in documention):
https://msdn.microsoft.com/cs-cz/library/system.net.sockets.tcpclient%28v=vs.110%29.aspx
When connection is closed .Read()/.Write() throws an exception. Does it mean that when .Write() method does not throw the data were delivered correctly to the other party or do I need to implement custom ACK logic?
I read documentation for both Socket and TcpClient class and none of them describe this case.

All that a returning send() call means (or any wrapper you use, like Socket or TcpClient) on a streaming, blocking internet socket is that the bytes are placed in the sending machine's buffer.
MSDN Socket.Send():
A successful completion of the Send method means that the underlying system has had room to buffer your data for a network send.
And:
The successful completion of a send does not indicate that the data was successfully delivered.
For .NET, the underlying implementation is WinSock2, documentation: send():
The successful completion of a send function does not indicate that the data was successfully delivered and received to the recipient. This function only indicates the data was successfully sent.
A call to send() returning does not mean the data was successfully delivered to the other side and read by the consuming application.
When data is not acknowledged in time, or when the other party sends a RST, the Socket (or whichever wrapper) will become in a faulted state, making the next send() or recv() fail.
So in order to answer your question:
Does it mean that when .Write() method does not throw the data were delivered
correctly to the other party or do I need to implement custom ACK logic?
No, it doesn't, and yes, you should - if it's important to your application that it knows another party has read that particular message.
This would for example be the case if a server-sent message indicates a state change of some sort on the client, which the client must apply to remain in sync. If the client doesn't acknowledge that message, the server cannot know for certain that the client has an up-to-date state.
In that case you could alter your protocol so that certain messages have a required response which the receiver must return. Do note that implementing an application protocol is surprisingly easy to do wrong. If you're inclined, you could implement having various protocol-dictated message flows using a state machine,
for both the server and the client.
Of course there are other solutions to that problem, such as giving each state a unique identifier, which is verified with the server before attempting any operation involving that state, triggering the retry of the earlier failed synchronization.
See also How to check the capacity of a TCP send buffer to ensure data delivery, Finding out if a message over tcp was delivered, C socket: does send wait for recv to end?

#CodeCaster's answer is correct and highlights the .NET documentation specifying the behavior of .Write(). Here is some complete test code to prove that he is right and the other answers saying things like "TCP guarantees message delivery" are unambiguously wrong:
using System;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Threading;
namespace TestEvents
{
class Program
{
static void Main(string[] args)
{
// Server: Start listening for incoming connections
const int PORT = 4411;
var listener = new TcpListener(IPAddress.Any, PORT);
listener.Start();
// Client: Connect to listener
var client = new TcpClient();
client.Connect(IPAddress.Loopback, PORT);
// Server: Accept incoming connection from client
TcpClient server = listener.AcceptTcpClient();
// Server: Send a message back to client to prove we're connected
const string msg = "We are now connected";
NetworkStream serverStream = server.GetStream();
serverStream.Write(ASCIIEncoding.ASCII.GetBytes(msg), 0, msg.Length);
// Client: Receive message from server to prove we're connected
var buffer = new byte[1024];
NetworkStream clientStream = client.GetStream();
int n = clientStream.Read(buffer, 0, buffer.Length);
Console.WriteLine("Received message from server: " + ASCIIEncoding.ASCII.GetString(buffer, 0, n));
// Client: Close connection and wait a little to make sure we won't ACK any more of server's messages
Console.WriteLine("Client is closing connection");
clientStream.Dispose();
client.Close();
Thread.Sleep(5000);
Console.WriteLine("Client has closed his end of the connection");
// Server: Send a message to client that client could not possibly receive
serverStream.Write(ASCIIEncoding.ASCII.GetBytes(msg), 0, msg.Length);
Console.WriteLine(".Write has completed on the server side even though the client will never receive the message. server.Client.Connected=" + server.Client.Connected);
// Let the user see the results
Console.ReadKey();
}
}
}
The thing to note is that execution proceeds normally all the way through the program and serverStream has no indication that the second .Write was not successful. This is despite the fact there is no way that second message can ever be delivered to its recipient. For a more detailed look at what's going on, you can replace IPAddress.Loopback with a longer route to your computer (like, have your router route port 4411 to your development computer and use the externally-visible IP address of your modem) and monitor that port in Wireshark. Here's what the output looks like:
Port 51380 is a randomly-chosen port representing the client TcpClient in the code above. There are double packets because this setup uses NAT on my router. So, the first SYN packet is my computer -> my external IP. The second SYN packet is my router -> my computer. The first PSH packet is the first serverStream.Write. The second PSH packet is the second serverStream.Write.
One might claim that the client does ACK at the TCP level with the RST packet, but 1) this is irrelevant to the use of TcpClient since that would mean TcpClient is ACKing with a closed connection and 2) consider what happens when the connection is completely disabled in the next paragraph.
If I comment out the lines that dispose the stream and close the client, and instead disconnect from my wireless network during the Thread.Sleep, the console prints the same output and I get this from Wireshark:
Basically, .Write returns without Exception even though no PSH packet was even dispatched, let alone had received an ACK.
If I repeat the process above but disable my wireless card instead of just disconnecting, THEN the second .Write throws an Exception.
Bottom line, #CodeCaster's answer is unambiguously correct on all levels and more than one of the other answers here are incorrect.

TcpClient uses TCP protocol which itself guarantees data delivery. If the data is not delivered, you will get an exception. If no exception is thrown - the data has been delivered.
Please see the description of the TCP protocol here: http://en.wikipedia.org/wiki/Transmission_Control_Protocol
Every time the data is sent, the sending computer waits for the acknowledgement packet to arrive, and if it has not arrived, it will re-try the send until it is either successful, timed out, or permanent network failure has been detected (for example, cable disconnect). In the latter two cases an exception will be thrown
Therefore, TCP offeres guaranteed data delivery in the sense that you always know whether the destination received your data or not
So, to answer your question, you do NOT need to implement custom ACK logic when using TcpClient, as it will be redundant

Guarantee is never possible. What you can know is that data has left your computer for delivery to other side in the order of data was sent.
If you want a very reliable system, you should implement acknowledgement logic by yourself based on your needs.

I agree with Denis.
From the documentation (and my experience): this method will block until all bytes were written or throw an exception on error (such as disconnect). If the method returns you are guaranteed that the bytes were delivered and read by the other side on the TCP level.
Vojtech - I think you missed the documentation since you need to look at the Stream you're using.
See: MSDN NetworkStream.Write method, in the remarks section:
The Write method blocks until the requested number of bytes is sent or a SocketException is thrown.
Notes
Assuring that the message was actually read properly by the listening application is another issue and the framework cannot guarantee this.
In async methods (such as BeginWrite or WriteAsync) it's a different ballgame since the method returns immediately and the mechanism to assure completion is different (EndWrite or task completion paired with a code).

how to check a remote udp port

I want to check the connection to a remote machine using UDPClient. Heard that it will return an icmp packet if failure occurs. How we can catch it?
How it is possible to check for a remote machine?
UdpClient receivingUdpClient = new UdpClient();
receivingUdpClient.Connect(IPAddress.Parse("10.2.2.13"), 80);
Byte[] sendBytes = Encoding.ASCII.GetBytes("0");
Var b=receivingUdpClient.Send(sendBytes, sendBytes.Length);

With UDP, there is no definite way of knowing whether the packet reached its destination or not (compare with TCP, which sends ack packets to let the sender know its packet was received).
It is true that in some cases, ICMP packets are sent, but what if the packet was filtered out (or simply dropped) somewhere along the routing path? As far as I have seen, most home routers are pre-configured to drop all ICMP on external ports, for example.
Instead of relying on ICMP packets, you could investigate if the protocol you are using has a PING packet (or some equivalent no-op packet, or if you created the protocol -- add it!) and use that with a timeout/retry logic to verify if the service is available.

Weird behavior of async tcp socket

I have an async server socket listening to some port. Then from another pc, I connect to the server and send 1 byte. Everything works fine, but there's a strange behavior. When I pull of network cable and try to send 1 byte (before os realizes cable was pulled of), I don't get any exception/error and as expected, server don't receive that packet. Is this how sockets are supposed to work? Does this mean that in case of connection loss some packets can be lost (because I don't get an exception and don't know that request was not sent)?
Here's the code:
private void button3_Click(object sender, EventArgs e)
{
var b = new byte[1] {1};
client.BeginSend(b, 0, b.Length, 0, new AsyncCallback(SendCallback), client);
}
private void SendCallback(IAsyncResult ar)
{
Socket client = (Socket)ar.AsyncState;
int bytesSent = client.EndSend(ar);
this.Invoke(new MethodInvoker(() => { MessageBox.Show(bytesSent.ToString() + " bytes sent"); }));
}

How could the sender possibly tell the packet was not received? It sent it out into a black hole and waits for reply. As long as there is no reply he cannot know whether the packet was received or never will be.
This is usually solved with timeouts. Eventually, the TCP stack will declare the connection dead, or your subsequent reads time out.
Sending does not guarantee delivery at all.
Have the other side send you a confirmation. Your confirmation read will time out eventually.
Or, Shutdown(Send) the socket. This ensures delivery and will throw an exception (after a timeout). You should Shutdown(Both) a socket anyway before closing it to make sure you get notified of all errors.

Windows at socket layer maintains a socket buffer at kernel. A successful send at application layer simply means the data is copied into the kernel buffer. The data in this buffer is pushed by the TCP stack to the remote application. In windows, if a packet is dropped, TCP resends the data for 3 times after which the TCP stack notifies the connection close to the application. Interval between retries is decided by the RTT. First retry is after 1*RTT, second is after 2*RTT & third after 3*RTT. In your case, the 1 byte you send simply copied into the kernel buffer & indicated success. It will take 3*RTT to indicate the socket is closed. This connection close is notified only if you invoke any socket API or monitoring the socket for close event. After pulling the cable, if you queue a second send exactly after 3*RTT, send should through the exception. Another way to get indication of the send failure immediately is by setting the send socket buffer size to zero (SetSocketOption(..,SendBuffer,..)) so that the TCP stack directly use your buffer & indicates a failure immediately.

I assume you created a TCP socket.
In that case, your Client will not be notified in case of disconnection (except if your application it sent some kind of logout message).
Looking at the Connected property of the TcpClient :
The Connected property gets the connection state of the Client socket
as of the last I/O operation. When it returns false, the Client socket
was either never connected, or is no longer connected.
"As of the last I/O" operation : only an (unsuccessful) Read from the Client will help you to detect the disconnection. Many applications implement "pings" to detect some disconnections.

Yes, this is the way TCP sockets are supposed to work. No, this does not mean that the packet is necessarily lost.
What is happening under the covers is this. When you call BeginSend, the byte you are sending is passed to the operating system's TCP stack. The TCP stack then sends the data in a packet.
When the packet is not acknowledged in a reasonable length of time, the TCP stack automatically resends the packet. This happens repeatedly as long as no acknowledgement packet is received.
If you plug the cable back in, one of these resends will get through, and the server will belatedly see the data. This is the desired behavior. Remember that TCP was designed to transmit military data during the cold war; the idea was that even if part of the network got nuked, the routing system would eventually adjust to find another path to the recipient, and the data would eventually get through.
If you don't plug the cable back in, most TCP stacks will eventually give up, terminating the connection. This takes several minutes, however.
More information on TCP retransmission timeout can be found in RFC 1122 here:
https://www.rfc-editor.org/rfc/rfc1122#page-95