I would like to send data over internet through a desktop application. I know a little bit about sockets. I have transferred the data within the LAN, but now I want to transfer the data over the internet. What is the best way to transfer both large and small quantities of data?
My system is connected to the server which has the access to the internet. My system's IP address is dynamic. I don't know how to send the data to another system which is connected to the internet. Do I need to find the router address? (My IP address is generated as 192.168.1.15).
Is using a socket enough, or is HTTP required?
Socket is enough if no firewalls/proxies are involved.
But, as Internet is involved (not the fastest connection), I suggest for the sake of convenience you should better opt for remoting over http. That way, even if in the future the setup changes, and firewalls/proxies get involved in the equation, you should not worry.
If all you want to do is transfer raw data from one machine to another it's very easy to do using a TCP socket.
Here's a quick example.
Server:
ThreadPool.QueueUserWorkItem(StartTCPServer);
private static void StartTCPServer(object state) {
TcpListener tcpServer = new TcpListener(IPAddress.Parse("192.168.1.15"), 5442);
tcpServer.Start();
TcpClient client = tcpServer.AcceptTcpClient();
Console.WriteLine("Client connection accepted from " + client.Client.RemoteEndPoint + ".");
StreamWriter sw = new StreamWriter("destination.txt");
byte[] buffer = new byte[1500];
int bytesRead = 1;
while (bytesRead > 0) {
bytesRead = client.GetStream().Read(buffer, 0, 1500);
if (bytesRead == 0) {
break;
}
sw.BaseStream.Write(buffer, 0, bytesRead);
Console.WriteLine(bytesRead + " written.");
}
sw.Close();
}
Client:
StreamReader sr = new StreamReader("source.txt");
TcpClient tcpClient = new TcpClient();
tcpClient.Connect(new IPEndPoint(IPAddress.Parse("192.168.1.15"), 5442));
byte[] buffer = new byte[1500];
long bytesSent = 0;
while (bytesSent < sr.BaseStream.Length) {
int bytesRead = sr.BaseStream.Read(buffer, 0, 1500);
tcpClient.GetStream().Write(buffer, 0, bytesRead);
Console.WriteLine(bytesRead + " bytes sent.");
bytesSent += bytesRead;
}
tcpClient.Close();
Console.WriteLine("finished");
Console.ReadLine();
More information about your connection needs is required in order to give you an appropriate solution. There are many protocols at your disposal and there are trade-offs for all of them. You will probably choose one of these two transport layers:
UDP - This is a send-and-forget method of sending packets. Good for streaming media that doesn't necessarily have to be 100% correct.
The good:
No connection required.
Very lightweight.
The bad:
No guarantee of your packet reaching the destination (although most of the time they make it).
Packets can arrive out of the order in which you sent them.
No guarantee that their contents are the same as when you sent the packet.
TCP - This is a connection-based protocol that guarantees predictable behavior.
The good:
You will know for sure whether the packet has reached the destination or not.
Packets will arrive in the order you sent them.
You are guaranteed that 99.999999999% of the time your packets will arrive with their contents unaltered.
Flow control - if the machine sending packets is sending too quickly, the receiving machine is able to throttle the sender's packet-sending rate.
The bad:
Requires a connection to be established.
Considerable more overhead than UDP.
The list of pros and cons is by no means complete but it should be enough information to give you the ability to make an informed decision. If possible, you should take advantage of application layer-based protocols that already exist, such as HTTP if you are transferring ASCII text, FTP if you are transferring files, and so on.
You can do it with .Net's Socket class or you can work with the more convenient TcpClient class.
Firstly though you need to figure out what server you intend to communicate with. Is it an HTTP server or an FTP server? Both HTTP and FTP are application-level protocols which are implemented on top of (using) sockets, which is really a transport layer interface.
Your local IP address or the address of the router really doesn't matter. You however need to know the IP address of the remote host you intend to connect to. You can obtain this by calling:
IPHostEntry host;
host = Dns.GetHostEntry(hostname);
You might also want to think about other issues when working with sockets, such as using timeouts to mask failure, the possibility of resuming upload/downloads when transferring large files, etc. If you spend sometime looking on the net, you should be able to find higher level HTTP/FTP apis that will let you work with file transfers much more easily.
Judging by your question, you seem pretty new to sockets, so reading this might also help
In your question you mix different things. Sockets are an abstraction for network communication. You will certainly need a socket to communicate over the network. However, possibly you will not see that a socket is used (like in a web-browser). Http is a communication protocol. This is what goes through a communication channel.
Visual Studio has a lot of well made facilities for creating and consuming SOAP XML Web Services. I'd look into it if I were you. Sure, there is some overhead, but coding against it is extremely easy.
Of course, I'm not sure how well that would scale if you had to transfer, say, tens or hundreads of megabytes of data across slow internet connections. It does offer asynchronous I/O, but I don't think you can get a progress indicator, and there most definately isn't a resume functionality.
Added: You can also continue using your socket. There is no extra work invloved for connecting to a server across the internet. Just specify the server's IP address, and away you go. Your OS will take care of all the gory details like routers, missing packets, etc.
First you should make a decision what protocol you want to use TCP or UDP. Then you have two options: 1. use Socket (lower level) or 2. Use class like TCPClient or UDPClient (which represents a little higher abstraction.
I'd suggest (for the begging the second option).
What you want to know depends heavily on many parts of your infrastructure.
If you want to send data to a server that is transparently connected
to the internet, it is as easy as connecting to it's IP adress.
If you want to connect to some friend with a broadband connection, things
get tricky. You usually have to configure both of your routers (or at least the
target one) for NAT.
Familiarize yourself with NAT, and the basics of IP routing.
The details you provided are not sufficient to describe exactly what
you want to do.
Related
I'm creating a screen sharing application. To allow for nat traversal I'm having to do this through a server, I'm just wondering how I can check that the client trying to connect to the server is the one I want to connect and not someone else, whilst not actually blocking other people from being able to connect to that port.
I'm thinking of using BeginAcceptTcpClient() instead of AcceptTcpClient() is there a way to check who is trying to connect before it accepts the connection this way so it can be picked up by the TcpListener later?
The only other thing I can of is to send the ID of the person I want to send the image to in the connection and let the server handle who to send it to but would that be too slow or resource intensive?
Maybe you should take a look on how join.me makes it:
On first machine start the Observable.
This connects to the central server and receives a unique id (ticket) that will be shown to the user.
On the second machine start the Observer.
This connects also to the central server and awaits an entering of the desired id to connect to.
Now the user on the observable machine has to communicate the id to the observer guy through a different channel (e.g. phone).
On the observer the ticket number is entered and the server can send both machines the ip address and port of each other where their did the hole punching. If a direct connect is possible, the central server has finished its work, otherwise both sides will send their data to the server and from their to the other client.
So to ensure that the correct clients are connected simply create a unique id which both sides have to share through a different communication channel.
I think the following should be enough to see if the correct client has connected to your server:
Create a TCPListener and start accepting connections either synchronously or async. I'm using the synchronous example from MSDN: http://msdn.microsoft.com/en-us/library/system.net.sockets.tcplistener.accepttcpclient(v=vs.110).aspx
When a client connects, create a stream to read data sent by the client like this:
NetworkStream stream = client.GetStream();
i = stream.Read(bytes, 0, bytes.Length);
Deserialize the message from bytes to string and check if it, for example, matches a unique ID that you are expecting:
`
data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
//There are, of course, better ways of doing this.
if(data.Equals("HandshakeId")
{
//start receiving/sending data in a loop
}
else
{
// Shutdown and end connection
client.Close();
}
`
A more interesting question is how to capture the actual screen you'll be sharing. I personally have done this in VS C++, and the performance was okay. There are several ways to do it, i.e. using GDI+, using DirectX or OpenGL, the list goes on..
I'm curious as to what are you planing to use :)
I'm writing a service that needs to maintain a long running SSL connection to a remote server. I need this server to be self-healing, that is if it's disconnected for any reason then the next time it's written to it will reconnect. I've tried this:
bool isConnected = client.Connected && client.Client.Poll(0, SelectMode.SelectWrite) && stream.CanWrite;
if (!isConnected )
{
this.connected = false;
GetConnection();
}
stream.Write(bytes, 0, bytes.Length);
stream.Flush();
But I find it doesn't act as I would expect it. If I simulate a network outage by disabling my wifi, I'm still able to write to the stream with stream.Write() for approximately 20 seconds. Then next time I try to write to it, none of client.Connected, client.Client.Poll(), or stream.CanWrite() return false, but when I go to write to the stream I get a socket exception. Finally, if I try to recreate the connection, I get this exception: An existing connection was forcibly closed by the remote host.
I would appreciate any help create a long running SslStream that can withstand network failure. Thanks!
From a 10.000 feet point of view:
The reason you can still write to the stream after shutting down your wifi is because there is a network buffer that is holding the data for transmission, stream.Write/stream.Flush success means the network interface (TCP/IP stack) has accepted the data and has been buffered for transmission, not that the data has reach its target.
It takes time to the TCP/IP Stack to notice a full media disconnection, (connection lost/reset) because even if there is no physical link TCP/IP will see this as a temporary issue in the network and will keep retrying for a while (the network could be dropping packets at some point and the stack will keep retrying)
If you think about this in the reverse way, you won't like all your programs to fail if there is a network hiccup (this happen too often on internet), so TCP/IP takes its time to notify to the app layer that the connection has become invalid (after retry several times and wait a reasonable amount of time)
You can always reconnect to the server when the SslStream fails and continue sending data, although you will find is not as easy as this because there are several scenarios where you send and data is not received by server and others where server receive the data and you do not receive any ACK from server at all... So depending on your needs, self-healing alone could be not enough.
Self-Healing is simple to implement, data consistency and reliability is harder and usually requires the server to be ready to support some kind of reliable messaging mechanism to ensure all data has been sent and received.
The underlying protocol for SSL is TCP. TCP will usually only send data if the application wants it to deliver data, or if it needs to reply to data received from the other side by sending an ACK. This means, that a broken connection like a lost link will not be noticed until you are trying to send any data. But you will not notice immediatly, because:
A write to the socket will only deliver the data to the OS kernel and return success if this delivery was successful.
The kernel will then try to deliver the data to the peer and will wait for the ACK from the client.
If it does not get any ACK it will retry again to deliver the data and only after some unsuccessful retries the kernel will declare the connection broken.
Only after the connection is marked broken by the kernel the next write or read will return the error from kernel to user space, like with returning EPIPE when doing a write.
This means, if you want to know up-front if the connection is still alive you have to make sure that you get a regular data exchange on the connection. At the TCP level you might set TCP_KEEPALIVE, but this might use an interval of some hours between exchanges packets. At the SSL layer you might try to use the infamous heartbeat extension, but most peers will not understand it. The last choice is to implement some kind of heartbeat in your own application.
As for the self healing: When reconnecting you get a new TCP connection and you also need to do a full SSL handshake, because the last SSL connection was not cleanly closed and thus cannot be resumed. The server has no idea that this new connection is just a continuation of the old one so you have to implement some kind of meta-connection spanning multiple TCP connections inside your application layer on both client and server. Inside this meta-connection you need to have your own data tracking to detect, which data are really accepted from the peer and which were only send but never explicitly accepted because the connection broke. Sound like a kind of TCP on top of TCP.
I have a server client application that I am currently working on. The server is receiving data fine over a WAN and the client seems to receive the data, but the client is only receiving one communication. Is there anything over a WAN that would make a client always only receive the first return UDP communication and none of the subsequent. Thanks for the help.
Client UDP Listening code
private void receiveUDP()
{
System.Net.IPEndPoint test = new System.Net.IPEndPoint(System.Net.IPAddress.Any,UDP_PORT_NUMBER);
System.Net.EndPoint serverIP = (System.Net.EndPoint)test;
server.Bind(serverIP);
//server.Ttl = 50;
EndPoint RemoteServ = (EndPoint)listenUDP;
do
{
byte[] content = new byte[1024];
int data = server.ReceiveFrom(content, ref RemoteServ);
string message = Encoding.ASCII.GetString(content);
ProcessCommands(message);
} while (true);
}
This is a bit of a stab in the dark (since you don't provide enough code to really say what's going on definitively), but there's one major reason why you might consistently see some UDP datagrams not be delivered over a WAN, while others always arrive successfully. This reason is MTU; the Maximum Transmission Unit which can be sent in a single UDP datagram. This can easily produce behaviour such as what you're seeing if (for example), your first datagram is a short "I accept your connection" message, and you then follow that with a datagrams containing large files; the first (small) datagram is smaller than the MTU and is delivered, while the following (large) datagrams are larger than the MTU, and are discarded en route.
For UDP over a WAN, the MTU will not be higher than about 1500 bytes, and in many situations may be as low as 1200 bytes. Any packets larger than that will be silently dropped somewhere between endpoints. To send large blocks of data via UDP, you need to chop them up into pieces smaller than the MTU for the network segment across which you're transmitting them.
On a LAN, you can usually get away with sending datagrams of any size. But as soon as they're being sent over the Internet or otherwise through heterogenous networks, they're likely to be silently discarded.
If you do need to send large files, you might choose to transmit them via TCP instead; TCP automatically manages chopping data up to fit within the MTU, and ensures that its packets are all received, and are received in order; guarantees that you will not receive from datagrams sent via UDP.
As I mentioned above, this is a complete stab in the dark and may not actually be related to your actual troubles. But it's the elephant in the room, when all we have to go on is that the first packet always arrives successfully, and later packets never do.
I've just started learning Sockets through various Google searches but I'm having some problems figuring it out how to properly use Sockets in C# and I'm in the need of some help.
I have a test application (Windows Forms) and on a different class (which is actually in it's own .dll, but that's irrelevant) I have all the server/client code for my sockets code.
Question 1)
On my test application, on the server part, the user can click the "start listening" button and the server part of my sockets application should start listening for connections on the specified address and port, so far so good.
However, the application will be blocked and I can't do anything until someone connects to the server. What if no one connects? How should I handle that? I could specify a receive timeout but what then? It throws an exception, what can I do with that? What I would like is to have some sort of activity on the main application so the user knows the application didn't froze and is waiting for connections. But if a connection doesn't come, it should timeout and close everything.
Maybe I should use asynchronous calls to send/receive methods but they seem confusing and I was not able to make it work, only synchronous work (I'll post my current code below).
Question 2)
Do I need to close anything when some send/receive call times out. As you'll see on my current code, I have a bunch of closes on the socket, but this doesn't feel right somehow. But it also doesn't feel right when an operation times out and I don't close the socket.
In conclusion of my two questions.... I would like an application that doesn't block so the user knows the server is waiting for a connection (with a little marquee animation for instance). If a connection is never established after a period of time, I want to close everything that should be closed. When a connection is established or if it doesn't happen after a period of time, I would like to inform the main application of the result.
Here's some of my code, the rest is similar. The Packet class is a custom class that represents my custom data unit, it's just a bunch of properties based on enums for now, with methods to convert them to bytes and back into properties.
The function that starts to listen for connections is something like this:
public void StartListening(string address, int port) {
try {
byte[] bufferBytes = new byte[32];
if(address.Equals("0.0.0.0")) {
udpSocket.Bind(new IPEndPoint(IPAddress.Any, port));
} else {
udpSocket.Bind(new IPEndPoint(IPAddress.Parse(address), port));
}
remoteEndPoint = new IPEndPoint(IPAddress.Any, 0);
int numBytesReceived = udpSocket.ReceiveFrom(bufferBytes, ref remoteEndPoint);
if(numBytesReceived == 0) {
udpSocket.Close();
return;
}
Packet syncPacket = new Packet(bufferBytes);
if(syncPacket.PacketType != PacketType.Control) {
udpSocket.Close();
return;
}
} catch {
if(udpSocket != null) {
udpSocket.Close();
}
}
}
I'm sure that I have a bunch of unnecessary code but I'm new at this and I'm not sure what do, any help fixing up my code and how to solve the issues above is really appreciated.
EDIT:
I should probably have stated that my requirements are to use UDP and implement these things myself in the application layer. You can consider this as homework but I haven't tagged as such because the code is irrelevant and will not be part of my grade and my problem (my question) is in "how to code" as my Sockets experience is minimal and it's not taught.
However I must say that I solved my problem for now I think... I was using threading on the demo application which was giving me some problems, now I'm using it in the protocol connections, makes more sense and I can easily change my custom protocol class properties and read those from the demo application.
I have specified a timeout and throw a SocketException if it reaches the timeout. Whenever an exception like this is caught, the socket connection is closed. I'm just talking about the connection handshake, nothing more. If no exceptions are caught, the code probably went smooth and the connection is established.
Please adapt your answers accordingly. Right now it doesn't make sense for me to marky any of them as the accepted answer, hope you understand.
You have got stuff a bit wrong.
First of all, UDP is connection-less. You do not connect or disconnect. All you do is to send and receive (must specify destination each time). You should also know that the only thing UDP promises is that a complete message arrives on each read. UDP do not guarantee that your messages arrive in the correct order or that they arrive at all.
TCP on the other hand is connection-based. You connect, send/receive and finally disconnect. TCP is stream-based (while UDP is message-based) which means that you can get a half message in the first read and the other half at the second read. TCP promises you that everything will arrive and in the correct order (or will die trying ;). So using TCP means that you should have some kind of logic to know when a complete message has arrived and a buffer that you use to build the complete message.
The next big question was about blocking. Since you are new at this, I recommend that you use Threads to handle sockets. Put the listener socket in one thread and each connecting socket in a separate thread (5 connected clients = 5 threads).
I also recommend that you use TCP since it's easier to build complete messages than ordering messages and build a transaction system (which will needed if you want to make sure that all messages arrives to/from clients).
Update
You still got UDP wrong. Close doesn't do anything other than cleaning up system resources. You should do something like this instead:
public void MySimpleServer(string address, int port)
{
try
{
byte[] bufferBytes = new byte[32];
if(address.Equals("0.0.0.0")) {
udpSocket.Bind(new IPEndPoint(IPAddress.Any, port));
} else {
udpSocket.Bind(new IPEndPoint(IPAddress.Parse(address), port));
}
remoteEndPoint = new IPEndPoint(IPAddress.Any, 0);
while (serverCanRun)
{
int numBytesReceived = udpSocket.ReceiveFrom(bufferBytes, ref remoteEndPoint);
// just means that one of the clients closed the connection using Shutdown.
// doesnt mean that we cant continue to receive.
if(numBytesReceived == 0)
continue;
// same here, loop to receive from another client.
Packet syncPacket = new Packet(bufferBytes);
if (syncPacket.PacketType != PacketType.Control)
continue;
HandlePacket(packet, endPoint);
}
} catch {
if(udpSocket != null) {
udpSocket.Close();
}
}
}
See? since there are no connection it's just waste of time to close a UDP socket to start listening from another one. The same socket can receive from ALL udp clients that know the correct port and address. That's what the remoteEndPoint is for. It tells which client that send the message.
Update 2
Small update to make a summary of all my comments.
UDP is connectionless. You can never detect if a connection have been established or disconnected. The Close method on a UDP socket will only free system resources. A call on client.Close() will not notify the server socket (as it will with TCP).
The best way to check if a connection is open is to create a ping/pong style of packet. i.e. the client sends a PING message and the server responds with a PONG. Remember that UDP will not try to resend your messages if they do not arrive. Therefore you need to resend the PING a couple of times before assuming that the server is down (if you do not receive a PONG).
As for clients closing you need to send your own message to the server telling it that the the client is going to stop talking to the server. For reliability the same thing goes here, keep resending the BYE message until you receive a reply.
imho it's mandatory that you implement a transactional system for UDP if you want reliability. SIP (google rfc3261) is an example of a protocol which uses transactions over UDP.
From your description I feel you should use TCP sockets instead of UDP. The difference is
TCP - You wait for a connection at a particuler IP:Port some user can connect to it and until the socket is closed can communicate by sending and receiveing information. This is like calling someone on phone.
UDP - you wait for a message at some IP:Port. User who wants to communicate just sends a message through UDP. You will receive the message through UDP. The order of delivery is not guaranteed. This is more like sending a snail mail to someone. There is no dedicated communication channel established.
Now coming to your problem
Server
Create a Socket with TCP family.
Either create a thread and accept the connection in that thread or use the BeginAccept apis of Socket.
In the main thread you can still display the ticker or whatever you want to do.
Client
Connect to the server.
Communicate by sending and receiving data.
I've read an example of a Tcp Echo Server and some things are unclear to me.
TcpClient client = null;
NetworkStream netStream = null;
try {
client = listener.AcceptTcpClient();
netStream = client.GetStream();
int totalBytesEchoed = 0;
while ((bytesRcvd = netStream.Read(rcvBuffer, 0, rcvBuffer.Length)) > 0) {
netStream.Write(rcvBuffer, 0, bytesRcvd);
totalBytesEchoed += bytesRcvd;
}
netStream.Close();
client.Close();
} catch {
netStream.Close();
}
When the server receives a packet (the while loop), he reads the data into rcvBuffer and writes it to the stream.
What confuses me is the chronological order of messages in communication. Is the data which was written with netStream.Write() sent immediately to the client (who may even still be sending), or only after the data which is already written to the stream (by client) processed.
The following question may even clarify the previous: If a client sends some data by writing to the stream, is that data moved to the message queue on the server side waiting to be read so the stream is actually "empty"? That would explain why the server can immediately write to stream - because the data which comes from the stream is actually buffered elsewhere...?
A TCP connection is, in principal, full duplex. So you are dealing with 2 separate channels and yes, both sides could be writing at the same time.
Hint: The method call NetworkStream.Read is blocking in that example.
The book is absolutely correct -- raw access to TCP streams does not imply any sort of extra "chunking" and, in this example for instance, a single byte could easily be processed at a time. However, performing the reading and writing in batches (normally with exposed buffers) can allow for more efficient processing (often as a result of less system calls). The network layer and network hardware also employ there own forms of buffers.
There is actually no guarantee that data written from Write() will actually be written before more Reads() successfully complete: even if data is flushed in one layer it does not imply it is flushed in another and there is absolutely no guarantee that the data has made its way back over to the client. This is where higher-level protocols come into play.
With this echo example the data is simply shoved through as fast as it can be. Both the Write and the Read will block based upon the underlying network stack (the send and receive buffers in particular), each with their own series of buffers.
[This simplifies things a bit of course -- one could always look at the TCP [protocol] itself which does impose transmission characteristics on the actual packet flow.]
You are right that technically when performing Read() operation, you are not reading bits off the wire. You are basically reading buffered data (chunks received by a TCP and arranged in a correct order). When sending you can Flush() that should in theory should send data immediately, but modern TCP stacks have a bit of logic how to gather data in appropriate size packets and burst them to the wire.
As Henk Holterman explained, TCP is a full duplex protocol (if supported by all underlying infrastructure), so sending and receiving data is more of when you server/client reads and writes data. It's not like when you server send data, a client will read it immediately. Client can be sending it's own data and then perform Read(), in this case data will stay in network buffer longer and can be discarded after some time it no-one want to read it. At least I've experienced this when dealing with my supa dupa server/client library (-.