I have created my client-server application but there is still something confusing about the whole process. What is packets? and what is the difference between sending a packet and sending a message (like i normally do).
So please, I expect a good explanation about:
What is packets?
How to structure them?
How to use them for communication in my client-server application?
For the sender and on a higher level a packet is equivalent to a message, you send packets or messages. On the receiving side a message may be split up into multiple packets. This splitting is most common using TCP connections, where you might have to do more than one receive-call (each returning a "packet") to receive a complete message.
To make matter more confusing, on a lower level a single TCP or UDP message (with TCP/IP headers prepended to the data) can also be called a packet. And on an even lower level an ethernet frame may be called a packet as well.
And to confuse even more, TCP connections has no real concept of message, it's just a stream of bytes. Messages are just an artificial concept on top of TCP. UDP on the other hand are distinct messages.
In short, it's probably simplest to just keep on using the term "message" when referring to the data you send or receive.
Related
How can you check if the message has reached the destination, I came up with a solution of mine but as I'm not a pro in this kind of topic I'd like to know some other ways.
My solutions is (client side) to send the packet, if no acknowledgement has been received within timeout time, then send once more, (server side) if the message received is correct send acknowledgement, if not received on the other side then send again.
This is a picture of the diagram of this algorithm,
Picture.
In short, both sides send the message twice.
any other ideas?
It depends on your application. But looking at the diagram you attached, you are preferring to TCP communication.
However if you really wanted to use UDP instead of TCP, you have to let go of ACK thing.
Assuming you are continuously streaming images to remote destination. And you do not worry about some frame loss as long as the streaming will be as fast as it will be. You can use UDP with that. Just also consider how reliable the transmission line (physical layer) to predict the outcome.
But if your application is not that time-critical but needs a highest reliability as much as possible, then you can use TCP.
For more detials [visit this]
Below are some comparison with UDP and TCP
The UDP protocol is an extremely simple protocol that allows for the sending of datagrams over IP. UDP is preferable to TCP for the delivery of time-critical data as many of the reliability features of TCP tend to come at the cost of higher latency and delays caused by the unconditional re-sending of data in the event of packet loss.
In contrast to TCP, which presents the programmer with one ordered octet stream per connected peer, UDP provides a packet-based interface with no concept of a connected peer. Datagrams arrive containing a source address1, and programmers are expected to track conceptual peer “connections” manually.
TCP guarantees2 that either a given octet will be delivered to the connected peer, or the connection will be broken and the programmer notified. UDP does not guarantee that any given packet will be delivered, and no notification is provided in the case of lost packets.
TCP guarantees that every octet sent will be received in the order that it was sent. UDP does not guarantee that transmitted packets will be received in any particular order, although the underlying protocols such as IP imply that packets will generally be received in the order transmitted in the absence of routing and/or hardware errors.
TCP places no limits on the size of transmitted data. UDP directly exposes the programmer to several implementation-specific (but also standardized) packet size limits. Creating packets of sizes that exceed these limits increase the chances that packets will either be fragmented or simply dropped. Fragmentation is undesirable because if any of the individual fragments of a datagram are lost, the datagram as a whole is automatically discarded. Working out a safe maximum size for datagrams is not trivial due to various overlapping standards.
Firstly I appreciate the UDP is not a reliable protocol, and I am not guaranteed to receive packets across a network.
However, if the packet does reach my machine, am I guaranteed to receive it at the application level, or can the network stack throw it away with impunity?
The reason I ask is that I seem to be missing packets occasionally, even though I know they're on the wire (simple EtherCAT bus, so packets always loop back).
No, there is no guarantee the packets will reach your application even if they reach your machine.
The kernel's UDP receive queue is finite, and if the packets arrive faster than your application can handle them, the queue will fill up and some of the packets will be dropped.
You can increase the size of the receive buffer (see this question), but you cannot make it unlimited.
I'm quite new to C# so please bear with me. I'm writing a relatively simple client server application in C# .NET 4.0. I am using TCP protocol, TCPListener and TCPClient to be more specific. I know how does TCP protocol work in theory. But I must be 100% sure that there will be no (unhandled) errors during data transfers.
After I send data how do I know whether data was successfully received. Can I totally rely on the underlying implementation of TCP protocol? So there is no need that I confirm from the other side that data was received?
It is crucial that I truly know which data was sent&successfully received.
I know it is a dumb question, but I really want to be sure. Thank you for your time and answers.
TCP guarantees that:
Data you send will arrive in the order you send it
Data you send will be received exactly as you sent it (unmodified)
No other (spurious) data will be received
It does not guarantee that rodents will not eat your cables, the power to the building will stay on, or even that the process on the other machine you are talking to will bother to do anything when the OS tells it that some data for it has arrived.
If you need a positive acknowledgement that data was received and acted upon, you need to send the acknowledgement back manually (since TCP connections are duplex, you already have a channel to do that).
Of course all of this does is not in any way specific to .NET, Windows, or any other implementation of a network stack.
Update: I 'd like to point out specifically that, after the OS network stack accepts data for transmission, there is no way for you to know that the process at the other end has received that data. The network stack knows in most cases that the data has reached the target (through TCP ACK messages), but it does not know if the OS on the target has fed them to the process they are destined for. So sending back your own "data received and acted upon" message is the only option.
With TCP/IP alone, you cannot determine whether any data is received - you would have to layer another protocol on top.
If you can find out if some particular data was received, then TCP/IP guarantees all data before that was also received.
There is no such thing as the '.NET TCP protocol'. There is just the TCP protocol, which has been running on the Internet for about 30 years. It's reliable. That was a design goal.
An option would be to use WCF Reliable Sessions over TCP. WCF reliable sessions will mask transport failures.
If you really need reliability AND network transport, why not use Message Queues? They have transactional guarantee (about as strong as the discipline of the developers working with it, but not stronger than that!)
It's like all your traffic with database-server-type safety (and ditto performance). I think you can easily configure .NET Remoting to use an MSMQ channel.
I've personally never done that, but I've used message queuing in general.
After sending some tcp data with the blocking/non-blocking methods such as:
Socket.Send() or Socket.SendAsync()
How can I know that my data has received an ACK message?
Can .NET know if TCP data has been successfully sent?
The only way to know for sure is to implement some kind of application-level acknowledgement. The TCP level "ACK" packet is not exposed to the application level at all, so you have to use something more than that.
You make the other end respond to it.
Even if TCP has Acked it, if the receiving end terminates (for good or bad reasons) before processing the message and acting on it, you still don't know, so the only way to know is for the other end to tell you.
This information isn't available from .net's class libraries. I had the same kind of considerations when I started working on this port scanner in C#. I have made use of a .NET wrapper for libpcap (after installing the corresponding driver), the SharpPcap (http://sourceforge.net/projects/sharppcap/), in order to get this kind of information. The ACK packets are obtained through SharpPcap's interface (invoking the native libpcap interface's transparently).
My application is NScanner Port Scanner/Sweeper and you can find the code at codeplex, referencing to you my simple usage of the aforementioned library (http://nscanner.codeplex.com/).
I hope I helped.
"I'm trying to focus on how can you know when your data has been accepted by the other-side of the connection."
I think you need to be aware what type of application layer protocol you are going to implement and what impact this has on application performance.
Take HTTP as an example of a "Streaming like" protocol. A server posts a stream of data to a client. There are no more additional application layer "ACKs" and the server doesn't actually care when and how exactly his stream of data arrives. This is very efficent on high latency links.
Now compare this to SMB! Instead of streaming a file, data is partitioned into blocks. Every successfully transferred block of data is acked on the application layer. This gives you more control, however, it effectively kills the protocol on WAN networks (check out "Bandwidth Delay Product").
Taking this into consideration, you can come up with your own design for your custom protocol.
The TCP layer will keep resending the packet until it receives a successful ACK.
Send will block until this happens - SendAsync will not block, and you can continue processing other stuff while the TCP layer handles sending the packet.
I recommend using Pcap.Net.
You can easily sniff packets using this library in C# and then easily check the packet values.
You can also easily build and send packets.
If you are really certain that you need to know the packet level details of your TCP connection, then in addition to creating the TCP socket for sending, you need your application to use the winpcap API to look at the raw traffic as well. You can install a filter to only receive packets relevant to the particular IP,port combination that determines your remote side.
There are a couple of projects out there creating .NET wrappers for libpcap, for example here
I am trying to develop a client/server application in which client will send images to the server.
Case 1:
Server is running on one machine that is behind a router and client is running on another machine that is behind some different router. As this communication will be on WAN (public IPs), a port is forwarded on server side router so that server can easily receive incoming UDP datagrams on that port.
UDP's maximum transmission unit (MTU) size is 64KB. It means a UDP socket should be able to transmit anything thing of size less than or equal to 65,536 bytes. When in case of the application i am developing the client is only able to send an image(UDP datagram) of 10-13k. If i try to transfer an image of the size greater than 10Kb, server is unable to receive it and server side UDP socket will be always in (receive) blocking mode.
Case 2:
Server is running on a machine that is behind a router and client is running on a machine that is behind the same router. It means client & server are on the same local area network. Even client and server are sharing the same local area network client is sending the images (UDP datagrams) on server's public IP. In this case server is able to receive any size of UDP datagram upto 64K, which is what i am expecting from my application.
I tried to run my client on different remote PCs but the result is same. Server is not able to receive a UDP datagram of bigger than 10-13Kb. If anyone can help me to deal with this situation, he would be much appreciated.
Link to the code:
http://pastebin.com/f644fee71
Thanks and goodluck with your projects.
Regards,
Atif
Although the IP layer may allow UDP packets of up to 64k in size I think you will find that the maximum "in the wild" UDP packet size will be limited to the smallest MTU of the devices in between your source and destination.
The standard Ethernet MTU is ~1500 bytes. Some devices support "jumbo" frames of up to ~10kbytes to improve performance. But this sort of thing isn't generally supported over the public internet, only on LANs.
The IP layer may fragment the UDP packet (unless the no-fragment bit is set in the packet). But the recipient will only receive and defragment the packet if every fragment is received in order (or out of order within a specific time limit). Otherwise it will discard the packet.
It may also be the case that not all the devices in between your source and destination support the frame size of the sending device. I've encountered situations where I needed to lower the MTU on my routers to ~1450 bytes because intermediate routers were discarding packets at 1500. This is due to MTU discovery not working reliably. Ie, the sending device has no way of determinging what the MTU is of devices on its path to the destination. Somewhere in that path a device will be discarding packets it considers too large.
UDP is a very bad idea for what you are doing. You would be better off using TCP.
If you are concerned about the performance of TCP connection setup/tear down then keep the connection up for as long as possible.
UDP is only a good protocol for delivering data when you don't care too much about whether the target receives the packet or not. Delivery is not guaranteed. In all other cases use TCP.
If you are determined to use UDP you will have to implement path MTU discovery in your protocol and prey that the routers/firewalls don't block the "fragmentation needed" ICMP packets. Which they shouldn't otherwise TCP wouldn't work either. But like I said, I've seen cases in the past where fragmentation needed ICMP packets are blocked or discarded and I had to manually tweak my own MTU.
UDP is a unreliable datagram protocol. The packets are not guaranteed to arrive at their destination, or in the order that you sent them either.
In addition, when sending packets larger than around 1500 bytes you'll find they get fragmented, or worse, dropped. The receiver will try to piece it together best it can.
But if anything is missing, goodbye packet. So really, the limit is actually around that 1500 byte mark but sometimes much less to ensure no fragmentation and that they arrive. To send data larger than that a higher level protocol is going to have to put them all back together for you and request anything thats missing.
How big will the images be? 64K might be too small anyway.
You have two or three options.
1) Use TCP - it's reliable and stream orientated. You don't have to worry about how big or small or fragmentation as this is taken care of for you.
2) Use UDP but develop a higher level application protocol on top. Depending on your protocol that could be quite a bit of work. I'm doing this right now though.
3) Have a look at using the UDT library. It's designed for transferring bulk data over a WAN and has better performance than TCP.
However, I would like to suggest that TCP will likely suit your needs just fine.