I recently stumbled across very strange bug. I first thought it was bug in my code, but right now I can see this is something system-wide, having said that, I think my problem fits SO best.
I was seeing very strange behavior of my network library and when I started debugging I discovered that if I will connect to a tcp socket, most often - soon after I started listening my sockets will get "stuck" in SYN_RCVD and SYN_SENT state. After even more debugging I dropped all my code altogether and I was left with this:
while (true)
{
TcpListener listener = new TcpListener(54000);
listener.Start();
var c = listener.AcceptTcpClientAsync();
TcpClient client = new TcpClient();
client.Connect(new IPEndPoint(IPAddress.Loopback, 54000));
c.Result.Close();
client.Close();
listener.Stop();
Console.WriteLine("done");
}
(classes above are System.Net.Sockets1s, not mine).
This code, on my machine will produce 1-5 dones and then will hang with 3 sockets: listening socket (bound to 0.0.0.0:54000 in listening state and 2 sockets, one stuck in SYN_RCVD, other one in SYN_SENT). I tried that on different machine, code runs ok, then I tried that on Linux VM on my 'broken' machine and it still does run ok.
Similar code in Node.js gets stuck as well.
Related
If I connect my pc directly to internet everything is ok, however with my router connected the following code get stuck in readline:
`
TcpListener webserver = new TcpListener(IPAddress.Any, 1302);
webserver.Start();
Debug.WriteLine("Started");
TcpClient client = webserver.AcceptTcpClient();
Debug.WriteLine("Client accepted");
sr = new StreamReader(client.GetStream());
sw = new StreamWriter(client.GetStream());
try
{
string req = sr.ReadLine();
Debug.WriteLine("Request: " + req);
if (req != null)
{
//HTTP/1.0 200 OK\n
//+read file to tcpstream
}`
I'm not trying to build a webserver I'm just testing things out with tcp.
If I use http://192.168.1.100:1302/ in a webbrowser or my public IP without the router the webpage loads so I'm guessing it's sg to do with my router or do I have to implement something else?
What I'm seeing is:
readline is stuck(If I refresh the page a few times the GET / HTTP/1.1 gets trough)
If I use read() I see a load of -1
I used wireshark and I'm seeing a lot of greyed out packages and a lot of retransmissions:
wireshark
I'm stuck first I thought I have to implement Upnp or nat punch through, but since some things are getting through I don't think establishing the connection is the problem.
Why are there packet losts? Where should I even start to debug the problem?
UPDATE #1:
The client(chrome webbrowser) is the same computer, but I tried outside computers(with and without behind a router) as well same result.
UPDATE #2: I think I might know what the problem is, I just tried an external webpage tester and there were no problems, so far every device was in the same ISP. I have to try it with vpn or a device that is outside of my ISP network.
I don't know if this is normal or not, but because of packet loss most packets never arrives to my application. This problem is defently caused by my router(and maybe sg. else), but I even tried to connect to my app with other devices in the same ISP same problem. I confirmed this by using vpn and also with another computer using different ISP.
Again, I need invaluable help of you.
I have project set up using HttpListener class to bind and listen on all found IPs and port(in this example) 1337.
Now, local connections(from my own computer typing IP:Port into address bar from browser) work just fine, but any connection from remote devices, even in the same wifi network, just don't hit listener.
Example code(instead of all found IPs I listen to all connections at port 1337)
HttpListener httplistener = new HttpListener();
httplistener.Prefixes.Add("http://+:80/");
httplistener.Start();
HttpListenerContext ctx = httplistener.GetContext();
Console.WriteLine("got context");
ctx.Response.OutputStream.Write(Encoding.ASCII.GetBytes("hello"), 0, 5);
Console.WriteLine("response written");
ctx.Response.Close();
Console.WriteLine("response closed");
Console.ReadKey();
Well, from any browser I get response as it should be(and under normal circumstances I use it I get JSON object for my JS app), but from any device in wifi network I got timeout and I never got even to the context.
TcpListener on the other hand works properly. Why? I know the difference more or less and in TcpListener I can get a raw connection(like from Putty):
TcpListener tcplistener = new TcpListener(endpoint);
tcplistener.Start();
TcpClient tcpclient = tcplistener.AcceptTcpClient();
int dataAvailable = tcpclient.Available;
Console.WriteLine("connection accepted");
using(StreamWriter sw = new StreamWriter(tcpclient.GetStream()))
{
sw.WriteLine("<html><body>hello</body></html>");
}
Console.WriteLine("response written");
tcpclient.Close();
Console.WriteLine("waiting for next connection");
Is it possible to somehow use HttpListener anyway or route connections to it? I really need possibility to connect to my app from every device through a html page. For any advices thank you. Tag #windows-phone-8.1 added because tried to connect from that device to HttpListener too with no positive effect.
Ok, case closed. After exhausting every possible option, sniffing packets revealed that send packets are lost in translation and never get to my computer where server is started. I pushed on IT and they declined everything but after confronting with hard evidence they had to yield and admit that someone made this specific subnetwork work in a way that separates every client from each other.
So, when I manually opened a socket I was able to connect(and sometimes not, but I thought it was just my wrong and messy implementation) but every other piece of traffic was mowed down before getting a chance to arrive at my network card.
I'm experiencing an issue specific to Windows 8 and VS2012.
I have a TCP socket server and client and am doing some testing on the local network. With sysinternals TCPView, I can see that packets are sent from the TCP client and arrive at the TCP Server (I see the packet counters increase).
However, it appears as if the data is not making it to the application stack?
The very same build runs without issues on Windows 7.
I have the Windows 8 firewall turned off and run both process with elevated permissions on a domain admin users with UAC turned off.
When I connect the client to a an outside server (running on a separate machine), everything works fine.
Is there anything else in Windows 8 that could prohibit TCP data communication between local processes?
Thanks,
EDIT
To make sure nothing in my server application is causing this issue, I built a quick TCP server in a console application, with the following code for the socket constructor:
listenerSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
and listen on the same local IP/Port as my server application.
I'm experiencing the same issue, I can telnet to the port but listenerSocket.AcceptAsync is never hit.
EDIT 2
Upon further testing, it appers my issue has something to do with the use of the Async socket calls, i.e. if I use the synchronous calls like socket.Accept(), the test application is performing normally. However, when I use Async socket calls, i.e. socket.AcceptAsync(), I'm experiencing the issues mentioned. So far I couldn't find any mention of differences between win7 & 8 in regards to async socket calls.
Here's my quick sample app that shows that the async callback is never triggered. This snippet works fine in Windows 7 but does not work in Windows 8 (try to telnet to 127.0.0.1 : 7000).
class Program
{
private static SocketAsyncEventArgs socketAsyncEventArgs = new SocketAsyncEventArgs();
static void Main(string[] args)
{
var listenerSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
listenerSocket.Bind(new IPEndPoint(IPAddress.Parse("127.0.0.1"), 7000));
listenerSocket.Listen(100);
socketAsyncEventArgs.Completed += AcceptEventArg_Completed;
listenerSocket.AcceptAsync(socketAsyncEventArgs);
Console.ReadLine();
}
private static void AcceptEventArg_Completed(object sender, SocketAsyncEventArgs e)
{
Console.WriteLine("AcceptEventArg_Completed");
}
}
EDIT 3
I found 2 others reporting the same issue on Microsoft Connect:
https://connect.microsoft.com/VisualStudio/feedback/details/759913/socketasynceventargs-completed-doesnt-fire-in-net-framework-4-5
and
http://connect.microsoft.com/VisualStudio/feedback/details/747218/saea-not-working-in-net-4-5-rp
Whereas the 2nd one is interesting as it seems to conclude there is a Windows bug in the Console.ReadLine() call and it is causing the issue and blocking the async callback. If I replace Console.ReadLine() in my snippet with:
while (true)
{
System.Threading.Thread.Sleep(10);
}
everything works fine.
See this: GetQueuedCompletionStatus can't dequeue IO from IOCP if the thread which originally issued the IO is blocking in ReadFile under windows 8
It's a bug in windows 8 and 2012 and affect all programs which used AcceptEx and ReadFile. As for now, only these two functions are known affected.
I meet the same thing when I was developing Tcp server and client applications with SocketAsyncEventArgs
I suggest you try this first.
open Windows firewall with Advanced Security
check the inbound / outbound rules to see if your application is blocked.
open AssemblyInfo.cs and change the
[assembly: Guid("06985fe3-80eb-48b4-940a-fd926e2f2053")]
to any other guid value.
By changing this, windows will think this is a new application and if there were any restrictions towards the old application, it wont be on the new one.
Sounds like this windows bug relating to the IOCP processing (possibly just of AcceptEx) on Windows 8 while other blocking I/O is in progress on the same thread:
http://social.technet.microsoft.com/Forums/en-GB/winserver8gen/thread/5764cd0f-fda1-4cfa-ae35-808210bae77e
So the socket connection is accepted, but your app never recieves notification of it.
Maybe Windows 8 does some weird, slighly broken, voodoo to convert synchronous IO like Console.Read into async internally.
You could just move your server code into a different thread, other workarounds to try might be to perform Accept synchonously or to change the console processing to be asynchronous (I can't really try that as I don't have Windows 8).
I've just started learning Sockets through various Google searches but I'm having some problems figuring it out how to properly use Sockets in C# and I'm in the need of some help.
I have a test application (Windows Forms) and on a different class (which is actually in it's own .dll, but that's irrelevant) I have all the server/client code for my sockets code.
Question 1)
On my test application, on the server part, the user can click the "start listening" button and the server part of my sockets application should start listening for connections on the specified address and port, so far so good.
However, the application will be blocked and I can't do anything until someone connects to the server. What if no one connects? How should I handle that? I could specify a receive timeout but what then? It throws an exception, what can I do with that? What I would like is to have some sort of activity on the main application so the user knows the application didn't froze and is waiting for connections. But if a connection doesn't come, it should timeout and close everything.
Maybe I should use asynchronous calls to send/receive methods but they seem confusing and I was not able to make it work, only synchronous work (I'll post my current code below).
Question 2)
Do I need to close anything when some send/receive call times out. As you'll see on my current code, I have a bunch of closes on the socket, but this doesn't feel right somehow. But it also doesn't feel right when an operation times out and I don't close the socket.
In conclusion of my two questions.... I would like an application that doesn't block so the user knows the server is waiting for a connection (with a little marquee animation for instance). If a connection is never established after a period of time, I want to close everything that should be closed. When a connection is established or if it doesn't happen after a period of time, I would like to inform the main application of the result.
Here's some of my code, the rest is similar. The Packet class is a custom class that represents my custom data unit, it's just a bunch of properties based on enums for now, with methods to convert them to bytes and back into properties.
The function that starts to listen for connections is something like this:
public void StartListening(string address, int port) {
try {
byte[] bufferBytes = new byte[32];
if(address.Equals("0.0.0.0")) {
udpSocket.Bind(new IPEndPoint(IPAddress.Any, port));
} else {
udpSocket.Bind(new IPEndPoint(IPAddress.Parse(address), port));
}
remoteEndPoint = new IPEndPoint(IPAddress.Any, 0);
int numBytesReceived = udpSocket.ReceiveFrom(bufferBytes, ref remoteEndPoint);
if(numBytesReceived == 0) {
udpSocket.Close();
return;
}
Packet syncPacket = new Packet(bufferBytes);
if(syncPacket.PacketType != PacketType.Control) {
udpSocket.Close();
return;
}
} catch {
if(udpSocket != null) {
udpSocket.Close();
}
}
}
I'm sure that I have a bunch of unnecessary code but I'm new at this and I'm not sure what do, any help fixing up my code and how to solve the issues above is really appreciated.
EDIT:
I should probably have stated that my requirements are to use UDP and implement these things myself in the application layer. You can consider this as homework but I haven't tagged as such because the code is irrelevant and will not be part of my grade and my problem (my question) is in "how to code" as my Sockets experience is minimal and it's not taught.
However I must say that I solved my problem for now I think... I was using threading on the demo application which was giving me some problems, now I'm using it in the protocol connections, makes more sense and I can easily change my custom protocol class properties and read those from the demo application.
I have specified a timeout and throw a SocketException if it reaches the timeout. Whenever an exception like this is caught, the socket connection is closed. I'm just talking about the connection handshake, nothing more. If no exceptions are caught, the code probably went smooth and the connection is established.
Please adapt your answers accordingly. Right now it doesn't make sense for me to marky any of them as the accepted answer, hope you understand.
You have got stuff a bit wrong.
First of all, UDP is connection-less. You do not connect or disconnect. All you do is to send and receive (must specify destination each time). You should also know that the only thing UDP promises is that a complete message arrives on each read. UDP do not guarantee that your messages arrive in the correct order or that they arrive at all.
TCP on the other hand is connection-based. You connect, send/receive and finally disconnect. TCP is stream-based (while UDP is message-based) which means that you can get a half message in the first read and the other half at the second read. TCP promises you that everything will arrive and in the correct order (or will die trying ;). So using TCP means that you should have some kind of logic to know when a complete message has arrived and a buffer that you use to build the complete message.
The next big question was about blocking. Since you are new at this, I recommend that you use Threads to handle sockets. Put the listener socket in one thread and each connecting socket in a separate thread (5 connected clients = 5 threads).
I also recommend that you use TCP since it's easier to build complete messages than ordering messages and build a transaction system (which will needed if you want to make sure that all messages arrives to/from clients).
Update
You still got UDP wrong. Close doesn't do anything other than cleaning up system resources. You should do something like this instead:
public void MySimpleServer(string address, int port)
{
try
{
byte[] bufferBytes = new byte[32];
if(address.Equals("0.0.0.0")) {
udpSocket.Bind(new IPEndPoint(IPAddress.Any, port));
} else {
udpSocket.Bind(new IPEndPoint(IPAddress.Parse(address), port));
}
remoteEndPoint = new IPEndPoint(IPAddress.Any, 0);
while (serverCanRun)
{
int numBytesReceived = udpSocket.ReceiveFrom(bufferBytes, ref remoteEndPoint);
// just means that one of the clients closed the connection using Shutdown.
// doesnt mean that we cant continue to receive.
if(numBytesReceived == 0)
continue;
// same here, loop to receive from another client.
Packet syncPacket = new Packet(bufferBytes);
if (syncPacket.PacketType != PacketType.Control)
continue;
HandlePacket(packet, endPoint);
}
} catch {
if(udpSocket != null) {
udpSocket.Close();
}
}
}
See? since there are no connection it's just waste of time to close a UDP socket to start listening from another one. The same socket can receive from ALL udp clients that know the correct port and address. That's what the remoteEndPoint is for. It tells which client that send the message.
Update 2
Small update to make a summary of all my comments.
UDP is connectionless. You can never detect if a connection have been established or disconnected. The Close method on a UDP socket will only free system resources. A call on client.Close() will not notify the server socket (as it will with TCP).
The best way to check if a connection is open is to create a ping/pong style of packet. i.e. the client sends a PING message and the server responds with a PONG. Remember that UDP will not try to resend your messages if they do not arrive. Therefore you need to resend the PING a couple of times before assuming that the server is down (if you do not receive a PONG).
As for clients closing you need to send your own message to the server telling it that the the client is going to stop talking to the server. For reliability the same thing goes here, keep resending the BYE message until you receive a reply.
imho it's mandatory that you implement a transactional system for UDP if you want reliability. SIP (google rfc3261) is an example of a protocol which uses transactions over UDP.
From your description I feel you should use TCP sockets instead of UDP. The difference is
TCP - You wait for a connection at a particuler IP:Port some user can connect to it and until the socket is closed can communicate by sending and receiveing information. This is like calling someone on phone.
UDP - you wait for a message at some IP:Port. User who wants to communicate just sends a message through UDP. You will receive the message through UDP. The order of delivery is not guaranteed. This is more like sending a snail mail to someone. There is no dedicated communication channel established.
Now coming to your problem
Server
Create a Socket with TCP family.
Either create a thread and accept the connection in that thread or use the BeginAccept apis of Socket.
In the main thread you can still display the ticker or whatever you want to do.
Client
Connect to the server.
Communicate by sending and receiving data.
On the client side I need to know when/if my socket connection has been broken. However the Socket.Connected property always returns true, even after the server side has been disconnected and I've tried sending data through it. Can anyone help me figure out what's going on here. I need to know when a socket has been disconnected.
Socket serverSocket = null;
TcpListener listener = new TcpListener(1530);
listener.Start();
listener.BeginAcceptSocket(new AsyncCallback(delegate(IAsyncResult result)
{
Debug.WriteLine("ACCEPTING SOCKET CONNECTION");
TcpListener currentListener = (TcpListener)result.AsyncState;
serverSocket = currentListener.EndAcceptSocket(result);
}), listener);
Socket clientSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
Debug.WriteLine("client socket connected: " + clientSocket.Connected);//should be FALSE, and it is
clientSocket.Connect("localhost", 1530);
Debug.WriteLine("client socket connected: " + clientSocket.Connected);//should be TRUE, and it is
Thread.Sleep(1000);
serverSocket.Close();//closing the server socket here
Thread.Sleep(1000);
clientSocket.Send(new byte[0]);//sending data should cause the socket to update its Connected property.
Debug.WriteLine("client socket connected: " + clientSocket.Connected);//should be FALSE, but its always TRUE
After doing some testing, it appears that the documentation for Socket.Connected is wrong, or at least misleading. clientSocket.Connected will only become false after clientSocket.close() is called. I think this is a throwback to the original C Berkeley sockets API and its terminology. A socket is bound when it has a local address associated with it, and a socket is connected when it has a remote address associated with it. Even though the remote side has closed the connection, the local socket still has the association and so it is still "connected".
However, here is a method that does work:
!(socket.Poll(0, SelectMode.SelectRead) && socket.Available == 0)
It relies on that fact that a closed connection will be marked as readable even though no data is available.
If you want to detect conditions such as broken network cables or computers abruptly being turned off, the situation is a bit more complex. Under those conditions, your computer never receives a packet indicating that the socket has closed. It needs to detect that the remote side has vanished by sending packets and noticing that no response comes back. You can do this at the application level as part of your protocol, or you can use the TCP KeepAlive option. Using TCP Keep Alive from .NET isn't particularly easy; you're probably better off building a keep-alive mechanism into your protocol (alternately, you could ask a separate question for "How do I enable TCP Keep Alive in .NET and set the keep alive interval?").
Just write to your socket as normal. You'll know when it's disconnected by the Exception that says your data couldn't be delivered.
If you don't have anything to write...then who cares if it's disconnected? It may be disconnected now, but come back before you need it - why bother tearing it down, and then looping a reconnect until the link is repaired...especially when you didn't have anything to say anyway?
If it bothers you, implement a keep alive in your protocol. Then you'll have something to say every 30 seconds or so.
Maybe solution is to send some dummy data through it and check if it times out?
I recommend stripping out the higher-level language stuff and explore what happens at the lower-level IO.
The lowest I've explored was while writing isectd (find on sourceforge). Using the select() system call, a descriptor for a closed socket becomes read-ready, and when isectd would attempt the recv() the socket's disconnected state can be confirmed.
As a solution, I recommend not writing your own socket IO and use someone else's middleware. There are lots of good candidates out there. Don't forget to consider simple queuing services as well.
PS. I would have provided URLs to all the above but my reputation (1) doesn't allow it.
does the clientSocket.Send() method wait for the packet to either be ack/nack'd?
If not your code is flying onto the next line while socket is still trying to figure out what is going on.