I have TCP server and clients written in C#. Since my connection is over wifi which is not reliable, I use resending the same packet and handle packet loss.
For example a bank account platform. The user deposites money and the client send this message to the server, if the server received this message, it will reply the client the operation is successful. If the client doesnt receive the reply, it will send again after a period of time.
This looks simple but I faced a situation when the wifi stucks and the client didnt receive reply and keep sending the same message to the server. End up those messages were received by the server at the same time. As a result the server thought the user deposites money 100 times.
I would like to know usually how people handle such case for tcp server client program, especially when the application is not just a chat application, but more sensitive information like money. My first thought is adding a transaction ID in the message so the server will not handle the messages with the same transaction ID, which will prevent the above case. But not sure if there is any better solution or .Net has some internal function for this.
Thank you.
When you code in C#, you are mostly working from within the Application layer of OSI model. TCP protocol works on the Transport layer (which is below the application layer).
Reliability, that you want to achieve, is already embedded inside the TCP protocol itself. This means, it will attempt to resent the packets, if some were lost, automatically without your additional requests. This will also happen before control is returned to the application layer program. There are also other guarantees, such as ordered delivery of the packets.
This means, that the functionality you need is already implemented at the layers bellow and you don't need to worry about it.
Note, if you were to use UDP, you would need to handle reliability problems yourself.
Related
I created extended TCP server and TCP client classes (in C#) for communication over network for my project use.
And as far as I understand, a client cannot really know if a server is down unless it requests for something which expects a reply but does not get it.
In our application, time and availability (of the server) are critical factors as it involve heavy machines for automation. Hence, according to the discussion on the design, the server is supposed to send its "Heart Beat" periodically such that in case a client does not receive anything from server after a period of time, it will:
Start to attempt its own recovery actions and if it still fails,
It will raise alarm to the service officer in the control room
I am supposed to implement the "heart beat" part in the server. And I have simple implementation of creating "Heart Beat".
public void SendHeartBeatToAllClients(byte[] hbdata) {
foreach (Socket socket in clientNoSocketList.Select(x => x.Value).ToList())
socket.Send(hbdata);
}
So far it works fine, but one thing that worries me is that the heart beat data (hbdata) is short (only few pre-arranged bytes, to save time to talk over many machines) and self-defined and since the server also sends some other data besides the hbdata, and considering the possible latency or other unexpected case, there is always a possibility for this hbdata to be mixed up. Also, in my "heart beat" implementation, the client does not need reply anything to the server.
So here are my questions:
Is my worry not well-grounded (as it is fine so far)? Is there any flaw?
Is Ping a better or a common way to have such heart beat functionality over TCP? Why or why not?
If Ping is to be implemented, considering that Ping has reply, is there a way to implement replyless Ping?
Any suggestion to make the heart beat robust enough yet in the shortest amount of data possible?
This is probably the hardest question to answer. Can you provide a little more detail? Why do you think that your server can't handle sending more than a few bytes? Are we talking thousands of machines here? Is everything on a local LAN, or does this go across multiple networks, or the internet?
Ping is an ICMP echo request - ping is very commonly used by networking monitor software, etc to ensure that clients are online. Typically you do not need to implement your own, if you are just pinging for network access (see: https://msdn.microsoft.com/en-us/library/system.net.networkinformation.ping(v=vs.110).aspx).
Also note that ping is not over TCP at all, but rather ICMP, a somewhat different protocol, used for network diagnostics among other things. But that brings me to number 3...
Ping without a reply is kind of pointless. For what you have in mind, I think the protocol you want is UDP - you can broadcast an arbitrary datagram, with no need for any kind of handshake or reply (TCP by definition involves establishing a session with a handshake) - it just sends. These would be Sockets with SocketType.Dgram instead of SocketType.Stream, and ProtocolType.Udp instead of Tcp or ICMP. If you want to get a little more involved, you can use Broadcast to send to same thing to the entire LAN, or Multicast to send to a specific group of clients.
Again, are you sure you need to be that concerned about limiting traffic, etc here?
Personally, I would flip it around, and have the clients "Check In" at a set interval, reporting a status code to the server. If the server notices a client hasn't checked in for a while, it should send a message to the client and expect a reply.
If you really are having issues scaling that up, I would have the server send the "Heart beats" via UDP at a set interval, and if the client thinks it's missing them, have a mechanism for it to hit the server and ask for a reply - and then if it doesn't get a response, raise the alarm.
Edit: just saw Prabhu's answer - he's right, ping will only tell you if the computer is up, you definitely want something inside the actual application to report back, not just the status of the network connection.
in my "heart beat" implementation, the client does not need reply anything to the server.
Application level keep-alives need to be two-way is'n't? What the above enables is that clients can be sure that server is alive and healthy on receiving the heart beat. If the client does not respond, server will not know the true status of the client. If client becomes unreachable,heart beats pile up in the servers send buffer. Server application will be oblivious to the fact.
Is my worry not well-grounded (as it is fine so far)? Is there any flaw?
Small sized bytes shouldn't be a problem. Its better the heart beats are small.
Is Ping a better or a common way to have such heart beat functionality over TCP? Why or why not?
Ping will be positive even if the client application is down but the system is healthy.
I'm writing a windows based client(c++) and server(c#) application which will communicate to each other via tcp packets. Here the client is sending data and server needs to acknowledge the same.
Now for this purpose I have made one single 'socket()' and 'connect()' call during the client lifetime on its startup. Some error checking and retries has been kept inside 'send()' and 'recv()' calling methods. Do note that one client will send one set (multiple packets) of data and quit at a time.
Now my questions are:
If the server is running continuously(e.g. windows service) on some PC, do I really need to consider about connection
breakdown(network failure) and creating a new socket and connect
accordingly from client?
If that be so, shall I need to consider resending the data from starting or from the point where client has failed to communicate last
time?
I want to know the general methods what people are using around the world for dealing this kind of situations for network applications.
do I really need to consider about connection breakdown and creating a new socket and connect accordingly from client?
Depends on how precious your data is. If you want to make sure it ended up at the server, and an error occurred while sending, then you can consider it "not sent".
If that be so, shall I need to consider resending the data from starting or from the point where client has failed to communicate last time?
That depends entirely on how your application logic and application protocol work. From your description we can't know how you send your data and how a server would recognize data it has already seen.
do I really need to consider about connection breakdown(network
failure) and creating a new socket and connect accordingly from
client?
You do certainly not need to create a new socket after connection shutdown; you can use the existing socket to connect anew.
So I am making this tcp program which simply sends and receives information between a client and a server.
My setup works as follows:
1)Server starts listening
2)Client sends "hello" command as well as a username/password
3)Server sends either "correctpass" or "wrongpass"
4)Client starts sending massive amounts of data in 50kb intervals
5)Server receives and stores this data as it comes
My question is: Is there something I should do to make sure that client doesn't send data when the server isn't listening? Forexample, should there be a command sent from server saying that it successfully got the data? I am just wondering this because I can't have the data come not in order.
I am receiving via tcp and I understand that TCP should send all the data to the server, but my problem is that the server might not be reading at the time that it is sent to it.
My other question is: Is TCP a good protocol for sending lots of small data (adding up to alot) through the internet? Is this how dropbox and other sync utilities communicate with their servers?
Edit:
I am currently using C# and networkstream to communicate
Thanks,
Rohit
First think that you need to do it's to read about data communications protocols and standarts thats already invented.
Includes OSI/ISO http://en.wikipedia.org/wiki/OSI_model
That help you to understand levels of tcp and udp, http, rest and etc.
Learn about technologies designed for interaction and communication like WCF.
But dont forget to play with your custom protocol it gives you experiences and representation how data comunications work and why and when use different protocols and technologies.
To work around data transfer collision you can use reqest/answer organization of communication.
But with WCF service you can do data transfer easyly. Without a lot of coding and misatkes.
Tcp is good to send data and be enshured from data coruption.
my problem is that the server might not be reading at the time that it
is sent to it.
The problem you are worrying about doesn't really exist. If the server doesn't have the connection open you will get a 'connection reset'. If the server isn't reading as fast as you are writing your writes will block in blocking mode, or return a retry indication in non-blocking mode.
I'm trying to build a simple multithreaded tcp server. The client connects and sends data to server, the server responds and waits for data again. The problem is I need the server to listen for incoming data in separate thread and be able to send command to client any time (for example to notify about new update). As far as I understood, when ever client sends data to server, if server doesn't respond with any data, client app doesn't let me send more data, server simply doesn't receive them. If I send data ether way around, does the data need to be 'acknowledged' for tcpclient?
Here's the source for the server: http://csharp.net-informations.com/communications/files/print/csharp-multi-threaded-server-socket_print.htm
How can I make the server send command to a client in separate thread outside the "DoChat" functions loop? or do I have to handle everything in that thread? Do I have to respond to each request client sends me? Thanks!
The problem is I need the server to listen for incoming data in separate thread
No, there is an async API. You can polll a list of threads to see which ahve new data waiting, obcviously to be done froa worker thread.
As far as I understood, when ever client sends data to server, if server doesn't respond with any
data, client app doesn't let me send more data, server simply doesn't receive them.
That is a lot more crap programming than the way sockets work. Sockets are totally ok with streaming ata in sending and receiving direction att the same time.
How can I make the server send command to a client in separate thread outside the "DoChat"
functions
Wel, me diong your job costs money.
BUT: The example is retarded. As in- totally anti pattern. One thread per client? You will run into memroy problems and perforamnce problems once 1000+ clients connect. You get tons of context switches.
Second, the client is not async because it is not written so. Mayy I suggest giong to the documentation, reading up on sockts an trying to build that yourself? THEN come back with questions that show more than "i just try to copy paste".
With proper programming this is totally normal. I have a similar application in development, sending data lall the time to the client and getting commands from the client to modify the data stream. Works liek a charm.
If I send data ether way around, does the data need to be 'acknowledged' for tcpclient?
Yes and no. No, not for TCP - TCP does it'Äs wn handshake under the hoods. Yes, if your protocol decides it has to, which is a programmer level design decision. It may or may not be necesssary, depending on the content of the data. Sometimes the acknowledgement provides more information (timestamp server side, tracking numer) and is not pure ly there for "I got it".
Users in field with PDA's will generate messages and send to the server; users at the server end will generate messages which need to be sent to the PDA.
Messages are between the app and server code; not 100% user entered data. Ie, we'll capture some data in a form, add GPS location, time date and such and send that to the server.
Server may send us messages like updates to database records used in the PDA app, messages for the user etc.
For messages from the PDA to server, that's easy. PDA initiates call to server and passes data. Presently using web services at the server end and "add new web reference" and associated code on the PDA.
I'm coming unstuck trying to get messages from the the server to the PDA in a timely fashion. In some instances receiving the message quickly is important.
If the server had a message for a particular PDA, it would be great for the PDA to receive that within a few seconds of it being available. So polling once a minute is out; polling once a second will generate a lot of traffic and, maybe draim the PDA battery some ?
This post is the same question as mine and suggests http long polling:
Windows Mobile 6.0/6.5 - Push Notification
I've looked into WCF callbacks and they appear to be exactly what I want however, unavailable for compact framework.
This next post isn't for CF but raises issues of service availability:
To poll or not to poll (in a web services context)
In my context i'll have 500-700 devices wanting to communicate with a small number of web services (between 2-5).
That's a lot of long poll requests to keep open.
Is sockets the way to go ? Again that's a lot of connections.
I've also read about methods using exchange or gmail; i'm really hesitant to go down those paths.
Most of the posts i've found here and in google are a few years old; something may have come up since then ?
What's the best way to handle 500-700 PDA CF devices wanting near-instant communication from a server, whilst maintaing battery life ? Tall request i'm sure.
Socket communication seems like the easiest approach. You say you're using webservices for client-server comms, and that is essentially done behind the scenes by the server (webservice) opening a socket and listening for packets arriving, then responding to those packets.
You want to take the same approach in reverse, so each client opens a socket on its machine and waits for traffic to arrive. The client will basically need to poll its own socket (which doesnt incur any network traffic). Client will also need to communicate its ip address and socket to the server so that when the server needs to communicate back to the client it has a means of reaching it. The server will then use socket based comms (as opposed to webservices) to send messages out as required. Server can just open a socket, send message, then close socket again. No need to have lots of permanently open sockets.
There are potential catches though if the client is roaming around and hopping between networks. If this is the case then its likely that the ip address will be changing (and client will need to open a new socket and pass the new ip address/socket info to the server). It also increases the chances that the server will fail to communicate with the client.
Sounds like an interesting project. Good luck!
Ages ago, the CF team built an application called the "Lunch Launcher" which was based on WCF store-and-forward messaging. David Kline did a nice series on it (here the last one, which has a TOC for all earlier articles).
There's an on-demand Webcast on MSDN given by Jim Wilson that gives an outline of store-and-forward and the code from that webcast is available here.
This might do what you want, though it got some dependencies (e.g. Exchange) and some inherent limitations (e.g. no built-in delivery confirmation).
Ok, further looking and I may be closer to what I want; which I think i a form of http long poll anyway.
This article here - http://www.codeproject.com/KB/IP/socketsincsharp.aspx - shows how to have a listener on a socket. So I do this on the server side.
Client side then opens a socket to the server at this port; sends it's device ID.
Server code first checks to see if there is a response for that device. If there is, it responds.
If not, it either polls itself or subscribes to some event; then returns when it's got data.
I could put in place time out code on the server side if needed.
Blocking on the client end i'm not worried about because it's a background thread and no data is the same as blocking at the app level; as to CPU & batter life, not sure.
I know what i've written is fairly broad, but is this a strategy worth exploring ?