I created extended TCP server and TCP client classes (in C#) for communication over network for my project use.
And as far as I understand, a client cannot really know if a server is down unless it requests for something which expects a reply but does not get it.
In our application, time and availability (of the server) are critical factors as it involve heavy machines for automation. Hence, according to the discussion on the design, the server is supposed to send its "Heart Beat" periodically such that in case a client does not receive anything from server after a period of time, it will:
Start to attempt its own recovery actions and if it still fails,
It will raise alarm to the service officer in the control room
I am supposed to implement the "heart beat" part in the server. And I have simple implementation of creating "Heart Beat".
public void SendHeartBeatToAllClients(byte[] hbdata) {
foreach (Socket socket in clientNoSocketList.Select(x => x.Value).ToList())
socket.Send(hbdata);
}
So far it works fine, but one thing that worries me is that the heart beat data (hbdata) is short (only few pre-arranged bytes, to save time to talk over many machines) and self-defined and since the server also sends some other data besides the hbdata, and considering the possible latency or other unexpected case, there is always a possibility for this hbdata to be mixed up. Also, in my "heart beat" implementation, the client does not need reply anything to the server.
So here are my questions:
Is my worry not well-grounded (as it is fine so far)? Is there any flaw?
Is Ping a better or a common way to have such heart beat functionality over TCP? Why or why not?
If Ping is to be implemented, considering that Ping has reply, is there a way to implement replyless Ping?
Any suggestion to make the heart beat robust enough yet in the shortest amount of data possible?
This is probably the hardest question to answer. Can you provide a little more detail? Why do you think that your server can't handle sending more than a few bytes? Are we talking thousands of machines here? Is everything on a local LAN, or does this go across multiple networks, or the internet?
Ping is an ICMP echo request - ping is very commonly used by networking monitor software, etc to ensure that clients are online. Typically you do not need to implement your own, if you are just pinging for network access (see: https://msdn.microsoft.com/en-us/library/system.net.networkinformation.ping(v=vs.110).aspx).
Also note that ping is not over TCP at all, but rather ICMP, a somewhat different protocol, used for network diagnostics among other things. But that brings me to number 3...
Ping without a reply is kind of pointless. For what you have in mind, I think the protocol you want is UDP - you can broadcast an arbitrary datagram, with no need for any kind of handshake or reply (TCP by definition involves establishing a session with a handshake) - it just sends. These would be Sockets with SocketType.Dgram instead of SocketType.Stream, and ProtocolType.Udp instead of Tcp or ICMP. If you want to get a little more involved, you can use Broadcast to send to same thing to the entire LAN, or Multicast to send to a specific group of clients.
Again, are you sure you need to be that concerned about limiting traffic, etc here?
Personally, I would flip it around, and have the clients "Check In" at a set interval, reporting a status code to the server. If the server notices a client hasn't checked in for a while, it should send a message to the client and expect a reply.
If you really are having issues scaling that up, I would have the server send the "Heart beats" via UDP at a set interval, and if the client thinks it's missing them, have a mechanism for it to hit the server and ask for a reply - and then if it doesn't get a response, raise the alarm.
Edit: just saw Prabhu's answer - he's right, ping will only tell you if the computer is up, you definitely want something inside the actual application to report back, not just the status of the network connection.
in my "heart beat" implementation, the client does not need reply anything to the server.
Application level keep-alives need to be two-way is'n't? What the above enables is that clients can be sure that server is alive and healthy on receiving the heart beat. If the client does not respond, server will not know the true status of the client. If client becomes unreachable,heart beats pile up in the servers send buffer. Server application will be oblivious to the fact.
Is my worry not well-grounded (as it is fine so far)? Is there any flaw?
Small sized bytes shouldn't be a problem. Its better the heart beats are small.
Is Ping a better or a common way to have such heart beat functionality over TCP? Why or why not?
Ping will be positive even if the client application is down but the system is healthy.
Related
I've seen many people talk about this any many people have criticised my attempt at doing this and said that I am not following the official rule of "What to send, who sends it, and how the other side responds." Is there even a rule for this? I'm not sure anymore...
Here is my current setup of communication.
Server: Gets alerted a new client has connected.
Server: Asks the client for the socket password.
Client: Sends a packet with the socket password.
Server: Okay, now give me some information on your device.
Client: Sends a packet including the device information
Server:
Okay, we've added you to the dictionary, thanks.
Now, if I tell the server to ask the client for the socket password straight away, what if the client hasn't called BeginReceive yet? What do I do about this?
I guess this question is answering my worries of the fact that I'm doing it wrong or I'm doing something wrong, how should I be doing this?
Who goes first? I've been told the client should only communicate with the server and the server should "respond" not "ask". Am I breaking any rules here?
Seegal. Ideally, you'd want to minimize the amount of calls the server is pinging down to the client. The server machine is your powerhouse, and if you're performing intensive work on a potato client PC, you may run into issues. Because you can't just tell people to upgrade their machines to use your service, that's on you.
The method I would use to hand up data would be a packet consisting of bitwise flags that will allow you to package multiple packet cases together without significant issue.
E.G.
[Flags]
enum NETWORK_CODES
{
CLIENT_CONNECT = 1 << 0,
CLIENT_DISCONNECT = 1 << 1,
CLIENT_PERFORM_ACTION = 1 << 2
}
This way, you can check for multiple flags at once and handle it accordingly.
if(packetFlags = CLIENT_CONNECT & CLIENT_PERFORM_ACTION) then
Authenticate(packetClient);
PerformAction();
end //Pseudo; don't hate. :D
It's a scalable solution, which is ideal for networking. If it doesn't scale, then you're in a bad spot and have to redo a whole bunch of code.
The server doesn't need to know if a client is there unless it's an active connection, which cuts back even further the amount of work you actually have to do.
There aren't any universal rules. Design your protocol according to your needs, and then you have rules. A socket is a peer to peer communication system (once the connection is established, in the case of TCP), which means that either peer may send any data at any time. It's purely up to the protocol engineer to design what can be sent, by whom, when, and how.
Think about Secure Sockets. SSL uses a protocol that allows the two peers to send and receive data through a secure channel however they wish, just like the underlying socket allows. On the other hand, think about HTTP. HTTP is purely a request/response oriented protocol. As such, it's far more restrictive than a protocol such as SSL, but it works perfectly well for the use cases it was designed for, along with many more, due to its inherent flexibility at the message level.
To answer the question "who goes first?", you can think of the act of connecting to the server as "going first". You can also think of the act of the server accepting the connection as "going second". So, that puts you back at square one; design your protocol however you see fit. Some protocols involve the server sending some sort of "welcome" message upon accepting the request, and some don't (HTTP, for example); either way is fine. In your case, it might make sense for the server to send a welcome message which contains some flags describing what is required to proceed, such as a password. The client would connect, consume the welcome message, and then proceed as appropriate.
To answer the question regarding whether or not the client has started reading from the socket, it isn't relevant. The server can send data immediately upon accepting the connection; the data will be buffered until the client application reads it by issuing read requests sufficient to consume it.
This is all more open-ended than is really appropriate for SO, but hopefully it helps.
I'm just making my server disconnect sockets that send no data after a certain amount of time, like 20 seconds.
I wonder whether working with timers is good for that or is there something special for that in socket library? Working with timers on the server for every socket makes it heavy.
Is it unsafe to make the client program handle that? For example every client disconnects after not sending data for a while.
This should be very easy to implement as part of your keep-alive checking. Unless you're completely ignoring the issue of dropped connections, you probably have a keep-alive system that periodically sends a message client->server and vice versa if there's been no communication. It should be trivial to add a simple "last data received time" value to the socket state, and then close the socket if it gets too far from DateTime.Now.
But the more important question is "Why?". The best solution depends on what your reasons for this are in the first place. Do you want to make the server usable to more clients by dumping those that aren't sending data? You'll probably make everything worse, since the timeouts for TCP sockets are more like 2-4 minutes, so when you disconnect the client after 20s and it reconnects, it will now be using two server-side ports, instead of one. Oops.
As for your comment on the deleted answer, and connection without data send and receive i think it gonna waste your threads points closer to your real problem - the amount of connection your server has should have no relation to how many threads the server uses to service those connections. So the only thing an open connection would "waste" is basically a bit of memory (depending on the amount of memory you need per connection, plus the socket cost with its buffers) and a TCP port. This can be an issue in some applications, but if you ever get to that level of "load", you can probably congratulate yourself already. You will much more likely run out of other resources before getting anywhere close to the port limits (assumption based on the fact that it sounds like you're making an MMO game). If you do really run into issues with those, you probably want to drop TCP anyway and rewrite everything in UDP (or preferrably, some ready solution on top of UDP).
The Client-Server model describes how a client should connect to a server and perform requests.
What I would recommend to you would be to connect to the server, and when you finish retrieving all the date you need, close the socket (on the client side).
The server will eventually find the socket's resources released, but you can check for the socket's Connected property to release the resources sooner.
When client disconnect to server then server can get disconnect event. it look like
socket.on('disconnect', function () {
// Disconnect event handling
});
On client side you also findout disconnect event .. in which you need to reconnect the server.
Users in field with PDA's will generate messages and send to the server; users at the server end will generate messages which need to be sent to the PDA.
Messages are between the app and server code; not 100% user entered data. Ie, we'll capture some data in a form, add GPS location, time date and such and send that to the server.
Server may send us messages like updates to database records used in the PDA app, messages for the user etc.
For messages from the PDA to server, that's easy. PDA initiates call to server and passes data. Presently using web services at the server end and "add new web reference" and associated code on the PDA.
I'm coming unstuck trying to get messages from the the server to the PDA in a timely fashion. In some instances receiving the message quickly is important.
If the server had a message for a particular PDA, it would be great for the PDA to receive that within a few seconds of it being available. So polling once a minute is out; polling once a second will generate a lot of traffic and, maybe draim the PDA battery some ?
This post is the same question as mine and suggests http long polling:
Windows Mobile 6.0/6.5 - Push Notification
I've looked into WCF callbacks and they appear to be exactly what I want however, unavailable for compact framework.
This next post isn't for CF but raises issues of service availability:
To poll or not to poll (in a web services context)
In my context i'll have 500-700 devices wanting to communicate with a small number of web services (between 2-5).
That's a lot of long poll requests to keep open.
Is sockets the way to go ? Again that's a lot of connections.
I've also read about methods using exchange or gmail; i'm really hesitant to go down those paths.
Most of the posts i've found here and in google are a few years old; something may have come up since then ?
What's the best way to handle 500-700 PDA CF devices wanting near-instant communication from a server, whilst maintaing battery life ? Tall request i'm sure.
Socket communication seems like the easiest approach. You say you're using webservices for client-server comms, and that is essentially done behind the scenes by the server (webservice) opening a socket and listening for packets arriving, then responding to those packets.
You want to take the same approach in reverse, so each client opens a socket on its machine and waits for traffic to arrive. The client will basically need to poll its own socket (which doesnt incur any network traffic). Client will also need to communicate its ip address and socket to the server so that when the server needs to communicate back to the client it has a means of reaching it. The server will then use socket based comms (as opposed to webservices) to send messages out as required. Server can just open a socket, send message, then close socket again. No need to have lots of permanently open sockets.
There are potential catches though if the client is roaming around and hopping between networks. If this is the case then its likely that the ip address will be changing (and client will need to open a new socket and pass the new ip address/socket info to the server). It also increases the chances that the server will fail to communicate with the client.
Sounds like an interesting project. Good luck!
Ages ago, the CF team built an application called the "Lunch Launcher" which was based on WCF store-and-forward messaging. David Kline did a nice series on it (here the last one, which has a TOC for all earlier articles).
There's an on-demand Webcast on MSDN given by Jim Wilson that gives an outline of store-and-forward and the code from that webcast is available here.
This might do what you want, though it got some dependencies (e.g. Exchange) and some inherent limitations (e.g. no built-in delivery confirmation).
Ok, further looking and I may be closer to what I want; which I think i a form of http long poll anyway.
This article here - http://www.codeproject.com/KB/IP/socketsincsharp.aspx - shows how to have a listener on a socket. So I do this on the server side.
Client side then opens a socket to the server at this port; sends it's device ID.
Server code first checks to see if there is a response for that device. If there is, it responds.
If not, it either polls itself or subscribes to some event; then returns when it's got data.
I could put in place time out code on the server side if needed.
Blocking on the client end i'm not worried about because it's a background thread and no data is the same as blocking at the app level; as to CPU & batter life, not sure.
I know what i've written is fairly broad, but is this a strategy worth exploring ?
I'm quite new to C# so please bear with me. I'm writing a relatively simple client server application in C# .NET 4.0. I am using TCP protocol, TCPListener and TCPClient to be more specific. I know how does TCP protocol work in theory. But I must be 100% sure that there will be no (unhandled) errors during data transfers.
After I send data how do I know whether data was successfully received. Can I totally rely on the underlying implementation of TCP protocol? So there is no need that I confirm from the other side that data was received?
It is crucial that I truly know which data was sent&successfully received.
I know it is a dumb question, but I really want to be sure. Thank you for your time and answers.
TCP guarantees that:
Data you send will arrive in the order you send it
Data you send will be received exactly as you sent it (unmodified)
No other (spurious) data will be received
It does not guarantee that rodents will not eat your cables, the power to the building will stay on, or even that the process on the other machine you are talking to will bother to do anything when the OS tells it that some data for it has arrived.
If you need a positive acknowledgement that data was received and acted upon, you need to send the acknowledgement back manually (since TCP connections are duplex, you already have a channel to do that).
Of course all of this does is not in any way specific to .NET, Windows, or any other implementation of a network stack.
Update: I 'd like to point out specifically that, after the OS network stack accepts data for transmission, there is no way for you to know that the process at the other end has received that data. The network stack knows in most cases that the data has reached the target (through TCP ACK messages), but it does not know if the OS on the target has fed them to the process they are destined for. So sending back your own "data received and acted upon" message is the only option.
With TCP/IP alone, you cannot determine whether any data is received - you would have to layer another protocol on top.
If you can find out if some particular data was received, then TCP/IP guarantees all data before that was also received.
There is no such thing as the '.NET TCP protocol'. There is just the TCP protocol, which has been running on the Internet for about 30 years. It's reliable. That was a design goal.
An option would be to use WCF Reliable Sessions over TCP. WCF reliable sessions will mask transport failures.
If you really need reliability AND network transport, why not use Message Queues? They have transactional guarantee (about as strong as the discipline of the developers working with it, but not stronger than that!)
It's like all your traffic with database-server-type safety (and ditto performance). I think you can easily configure .NET Remoting to use an MSMQ channel.
I've personally never done that, but I've used message queuing in general.
I am about to develop a Network measurement tool. The objective is to make a tool, which can measure the responsetime in between a client and a server machine (from the client side). It is s side-application to a main application - If the main applicaiton experiences that the responsetime from the server is above a certain threshold, the tool will be kicked alive, and performs network connectivity tests, to determine of the client server connection is stable (it might be unstable, due to the network being wireless etc.)
The tests I need to perform are not just ping operations, but also transmitting packages of different size.
I have however very little experience in communications technology.
Is ICMP protocol the way to go? and if yes, is it possible to send packages of differnet sizes (to measure if the network is able to transfer eg. 2 MB of data in a reasonable time)?
I have a second concern. What should I look out for in regards to firewalls? It would be a shame to develop an application which works fine on my local network, but as soon as it is used out in the real life, it fails misserably because the tests are blocked by a firewall.
I hope my questions aren't too noobish, but know that any help is much appreciated.
All the best
/Sagi
To keep clear of firewalls, you should do a test using the same protocol and port you use, and create inside of your application a new type of message that should be responded as soon as it is read by the server: You should program your ping measures.
Then the client would measure the times spent in travel traveled and compute your ping and relay it back to your server. This also gives a better reading when in case of some ISPs that give a ICMP protocol packets a huge advantage over other packages on their QoS server, artificially creating(faking) lower latency. And also, you would not have to worry about the firewall not allowing your ICMP packets, because you would have to be allowed to conect on the standart port you use.
Also, most games work this way (Half-Life, Age Of Empires etc.) , and not by sending standard Ping packets.