I'm quite new to C# so please bear with me. I'm writing a relatively simple client server application in C# .NET 4.0. I am using TCP protocol, TCPListener and TCPClient to be more specific. I know how does TCP protocol work in theory. But I must be 100% sure that there will be no (unhandled) errors during data transfers.
After I send data how do I know whether data was successfully received. Can I totally rely on the underlying implementation of TCP protocol? So there is no need that I confirm from the other side that data was received?
It is crucial that I truly know which data was sent&successfully received.
I know it is a dumb question, but I really want to be sure. Thank you for your time and answers.
TCP guarantees that:
Data you send will arrive in the order you send it
Data you send will be received exactly as you sent it (unmodified)
No other (spurious) data will be received
It does not guarantee that rodents will not eat your cables, the power to the building will stay on, or even that the process on the other machine you are talking to will bother to do anything when the OS tells it that some data for it has arrived.
If you need a positive acknowledgement that data was received and acted upon, you need to send the acknowledgement back manually (since TCP connections are duplex, you already have a channel to do that).
Of course all of this does is not in any way specific to .NET, Windows, or any other implementation of a network stack.
Update: I 'd like to point out specifically that, after the OS network stack accepts data for transmission, there is no way for you to know that the process at the other end has received that data. The network stack knows in most cases that the data has reached the target (through TCP ACK messages), but it does not know if the OS on the target has fed them to the process they are destined for. So sending back your own "data received and acted upon" message is the only option.
With TCP/IP alone, you cannot determine whether any data is received - you would have to layer another protocol on top.
If you can find out if some particular data was received, then TCP/IP guarantees all data before that was also received.
There is no such thing as the '.NET TCP protocol'. There is just the TCP protocol, which has been running on the Internet for about 30 years. It's reliable. That was a design goal.
An option would be to use WCF Reliable Sessions over TCP. WCF reliable sessions will mask transport failures.
If you really need reliability AND network transport, why not use Message Queues? They have transactional guarantee (about as strong as the discipline of the developers working with it, but not stronger than that!)
It's like all your traffic with database-server-type safety (and ditto performance). I think you can easily configure .NET Remoting to use an MSMQ channel.
I've personally never done that, but I've used message queuing in general.
Related
I've seen many people talk about this any many people have criticised my attempt at doing this and said that I am not following the official rule of "What to send, who sends it, and how the other side responds." Is there even a rule for this? I'm not sure anymore...
Here is my current setup of communication.
Server: Gets alerted a new client has connected.
Server: Asks the client for the socket password.
Client: Sends a packet with the socket password.
Server: Okay, now give me some information on your device.
Client: Sends a packet including the device information
Server:
Okay, we've added you to the dictionary, thanks.
Now, if I tell the server to ask the client for the socket password straight away, what if the client hasn't called BeginReceive yet? What do I do about this?
I guess this question is answering my worries of the fact that I'm doing it wrong or I'm doing something wrong, how should I be doing this?
Who goes first? I've been told the client should only communicate with the server and the server should "respond" not "ask". Am I breaking any rules here?
Seegal. Ideally, you'd want to minimize the amount of calls the server is pinging down to the client. The server machine is your powerhouse, and if you're performing intensive work on a potato client PC, you may run into issues. Because you can't just tell people to upgrade their machines to use your service, that's on you.
The method I would use to hand up data would be a packet consisting of bitwise flags that will allow you to package multiple packet cases together without significant issue.
E.G.
[Flags]
enum NETWORK_CODES
{
CLIENT_CONNECT = 1 << 0,
CLIENT_DISCONNECT = 1 << 1,
CLIENT_PERFORM_ACTION = 1 << 2
}
This way, you can check for multiple flags at once and handle it accordingly.
if(packetFlags = CLIENT_CONNECT & CLIENT_PERFORM_ACTION) then
Authenticate(packetClient);
PerformAction();
end //Pseudo; don't hate. :D
It's a scalable solution, which is ideal for networking. If it doesn't scale, then you're in a bad spot and have to redo a whole bunch of code.
The server doesn't need to know if a client is there unless it's an active connection, which cuts back even further the amount of work you actually have to do.
There aren't any universal rules. Design your protocol according to your needs, and then you have rules. A socket is a peer to peer communication system (once the connection is established, in the case of TCP), which means that either peer may send any data at any time. It's purely up to the protocol engineer to design what can be sent, by whom, when, and how.
Think about Secure Sockets. SSL uses a protocol that allows the two peers to send and receive data through a secure channel however they wish, just like the underlying socket allows. On the other hand, think about HTTP. HTTP is purely a request/response oriented protocol. As such, it's far more restrictive than a protocol such as SSL, but it works perfectly well for the use cases it was designed for, along with many more, due to its inherent flexibility at the message level.
To answer the question "who goes first?", you can think of the act of connecting to the server as "going first". You can also think of the act of the server accepting the connection as "going second". So, that puts you back at square one; design your protocol however you see fit. Some protocols involve the server sending some sort of "welcome" message upon accepting the request, and some don't (HTTP, for example); either way is fine. In your case, it might make sense for the server to send a welcome message which contains some flags describing what is required to proceed, such as a password. The client would connect, consume the welcome message, and then proceed as appropriate.
To answer the question regarding whether or not the client has started reading from the socket, it isn't relevant. The server can send data immediately upon accepting the connection; the data will be buffered until the client application reads it by issuing read requests sufficient to consume it.
This is all more open-ended than is really appropriate for SO, but hopefully it helps.
I created extended TCP server and TCP client classes (in C#) for communication over network for my project use.
And as far as I understand, a client cannot really know if a server is down unless it requests for something which expects a reply but does not get it.
In our application, time and availability (of the server) are critical factors as it involve heavy machines for automation. Hence, according to the discussion on the design, the server is supposed to send its "Heart Beat" periodically such that in case a client does not receive anything from server after a period of time, it will:
Start to attempt its own recovery actions and if it still fails,
It will raise alarm to the service officer in the control room
I am supposed to implement the "heart beat" part in the server. And I have simple implementation of creating "Heart Beat".
public void SendHeartBeatToAllClients(byte[] hbdata) {
foreach (Socket socket in clientNoSocketList.Select(x => x.Value).ToList())
socket.Send(hbdata);
}
So far it works fine, but one thing that worries me is that the heart beat data (hbdata) is short (only few pre-arranged bytes, to save time to talk over many machines) and self-defined and since the server also sends some other data besides the hbdata, and considering the possible latency or other unexpected case, there is always a possibility for this hbdata to be mixed up. Also, in my "heart beat" implementation, the client does not need reply anything to the server.
So here are my questions:
Is my worry not well-grounded (as it is fine so far)? Is there any flaw?
Is Ping a better or a common way to have such heart beat functionality over TCP? Why or why not?
If Ping is to be implemented, considering that Ping has reply, is there a way to implement replyless Ping?
Any suggestion to make the heart beat robust enough yet in the shortest amount of data possible?
This is probably the hardest question to answer. Can you provide a little more detail? Why do you think that your server can't handle sending more than a few bytes? Are we talking thousands of machines here? Is everything on a local LAN, or does this go across multiple networks, or the internet?
Ping is an ICMP echo request - ping is very commonly used by networking monitor software, etc to ensure that clients are online. Typically you do not need to implement your own, if you are just pinging for network access (see: https://msdn.microsoft.com/en-us/library/system.net.networkinformation.ping(v=vs.110).aspx).
Also note that ping is not over TCP at all, but rather ICMP, a somewhat different protocol, used for network diagnostics among other things. But that brings me to number 3...
Ping without a reply is kind of pointless. For what you have in mind, I think the protocol you want is UDP - you can broadcast an arbitrary datagram, with no need for any kind of handshake or reply (TCP by definition involves establishing a session with a handshake) - it just sends. These would be Sockets with SocketType.Dgram instead of SocketType.Stream, and ProtocolType.Udp instead of Tcp or ICMP. If you want to get a little more involved, you can use Broadcast to send to same thing to the entire LAN, or Multicast to send to a specific group of clients.
Again, are you sure you need to be that concerned about limiting traffic, etc here?
Personally, I would flip it around, and have the clients "Check In" at a set interval, reporting a status code to the server. If the server notices a client hasn't checked in for a while, it should send a message to the client and expect a reply.
If you really are having issues scaling that up, I would have the server send the "Heart beats" via UDP at a set interval, and if the client thinks it's missing them, have a mechanism for it to hit the server and ask for a reply - and then if it doesn't get a response, raise the alarm.
Edit: just saw Prabhu's answer - he's right, ping will only tell you if the computer is up, you definitely want something inside the actual application to report back, not just the status of the network connection.
in my "heart beat" implementation, the client does not need reply anything to the server.
Application level keep-alives need to be two-way is'n't? What the above enables is that clients can be sure that server is alive and healthy on receiving the heart beat. If the client does not respond, server will not know the true status of the client. If client becomes unreachable,heart beats pile up in the servers send buffer. Server application will be oblivious to the fact.
Is my worry not well-grounded (as it is fine so far)? Is there any flaw?
Small sized bytes shouldn't be a problem. Its better the heart beats are small.
Is Ping a better or a common way to have such heart beat functionality over TCP? Why or why not?
Ping will be positive even if the client application is down but the system is healthy.
I have TCP server and clients written in C#. Since my connection is over wifi which is not reliable, I use resending the same packet and handle packet loss.
For example a bank account platform. The user deposites money and the client send this message to the server, if the server received this message, it will reply the client the operation is successful. If the client doesnt receive the reply, it will send again after a period of time.
This looks simple but I faced a situation when the wifi stucks and the client didnt receive reply and keep sending the same message to the server. End up those messages were received by the server at the same time. As a result the server thought the user deposites money 100 times.
I would like to know usually how people handle such case for tcp server client program, especially when the application is not just a chat application, but more sensitive information like money. My first thought is adding a transaction ID in the message so the server will not handle the messages with the same transaction ID, which will prevent the above case. But not sure if there is any better solution or .Net has some internal function for this.
Thank you.
When you code in C#, you are mostly working from within the Application layer of OSI model. TCP protocol works on the Transport layer (which is below the application layer).
Reliability, that you want to achieve, is already embedded inside the TCP protocol itself. This means, it will attempt to resent the packets, if some were lost, automatically without your additional requests. This will also happen before control is returned to the application layer program. There are also other guarantees, such as ordered delivery of the packets.
This means, that the functionality you need is already implemented at the layers bellow and you don't need to worry about it.
Note, if you were to use UDP, you would need to handle reliability problems yourself.
I have the same dilemma as the one who posted this topic, Real-time communication with WCF
except that my problem is not about games programming. I would like to know what's the best method to use to be able to have a real time communication in between two windows applications (server-client). I am using visual c++/c# to date and i would like to be able to display all the Feeds that are being received by my server to the client in real time.
I have started trying to use .NET remoting but in my continuous research, it appears that it will use SOAP-http, and might affect the speed of the communication. My server and client will communicate using the internet and .NET remoting does not permit the use of TCP Channel when communicating in between a firewall or the internet.
Your inputs will be greatly appreciated.
I guess it depends on your escenario, if you want "real-time" and you are willing to lose some packages in the process you are better with UDP, take a video conferencing tool for example, by the time you recover your slow packages you will have to move and display the next frame in the video or audio; that is a good example for the use of UDP. This is the reason why UDP is much faster than TCP.
If however, you are not willing to lose a single bit of your message, then TCP was made for you because if you lost a package the protocol will request it again to have your complete message as complete as possible.
Additionally it depends on the way the communication is being sustained, is the information flowing from one to many?, from many to many?, one to tone?
Take NetNamedPipeBinding for instance, this will be much faster process, but is only deployed in a single machine but accross processes. Whereas NetMsmqBinding will help you to build queues and it will be amazingly reliable and scalable for scenarios where your load will be a massive number of connections.
In the end, it all boils down to your concrete escenario and your business goals.
Hope it helps
If you are willing to do your own message parsing, you can use standard TCP sockets with the TcpClient and TcpListener classes. If your data is already a serializable object, you could serialize it into a text stream and just send it over the socket, deserializing it on the client side.
To get it to work over the internet, the server needs to have the port forwarded on your router, the client would just attach to the server's public IP. You would obviously need to add an exception in your firewall for this port as well.
The biggest problem with WCF and large data is setting up the streaming, by default WCF sends everything at once, which isn't practical for large files.
After sending some tcp data with the blocking/non-blocking methods such as:
Socket.Send() or Socket.SendAsync()
How can I know that my data has received an ACK message?
Can .NET know if TCP data has been successfully sent?
The only way to know for sure is to implement some kind of application-level acknowledgement. The TCP level "ACK" packet is not exposed to the application level at all, so you have to use something more than that.
You make the other end respond to it.
Even if TCP has Acked it, if the receiving end terminates (for good or bad reasons) before processing the message and acting on it, you still don't know, so the only way to know is for the other end to tell you.
This information isn't available from .net's class libraries. I had the same kind of considerations when I started working on this port scanner in C#. I have made use of a .NET wrapper for libpcap (after installing the corresponding driver), the SharpPcap (http://sourceforge.net/projects/sharppcap/), in order to get this kind of information. The ACK packets are obtained through SharpPcap's interface (invoking the native libpcap interface's transparently).
My application is NScanner Port Scanner/Sweeper and you can find the code at codeplex, referencing to you my simple usage of the aforementioned library (http://nscanner.codeplex.com/).
I hope I helped.
"I'm trying to focus on how can you know when your data has been accepted by the other-side of the connection."
I think you need to be aware what type of application layer protocol you are going to implement and what impact this has on application performance.
Take HTTP as an example of a "Streaming like" protocol. A server posts a stream of data to a client. There are no more additional application layer "ACKs" and the server doesn't actually care when and how exactly his stream of data arrives. This is very efficent on high latency links.
Now compare this to SMB! Instead of streaming a file, data is partitioned into blocks. Every successfully transferred block of data is acked on the application layer. This gives you more control, however, it effectively kills the protocol on WAN networks (check out "Bandwidth Delay Product").
Taking this into consideration, you can come up with your own design for your custom protocol.
The TCP layer will keep resending the packet until it receives a successful ACK.
Send will block until this happens - SendAsync will not block, and you can continue processing other stuff while the TCP layer handles sending the packet.
I recommend using Pcap.Net.
You can easily sniff packets using this library in C# and then easily check the packet values.
You can also easily build and send packets.
If you are really certain that you need to know the packet level details of your TCP connection, then in addition to creating the TCP socket for sending, you need your application to use the winpcap API to look at the raw traffic as well. You can install a filter to only receive packets relevant to the particular IP,port combination that determines your remote side.
There are a couple of projects out there creating .NET wrappers for libpcap, for example here