We Keep Coding

TCPClient is not closed, and instead waits for a client to close it

i have few questions about programming a TcpListener.
First problem:
Once client is connected using browser, i see the request. it is all ok. but then i face the problem with writing. and client recieving that data. it basically never gets a reply from server. do i need that flush function ? how does it work ? and is there any others ways of doing it ?
Porblem number 2 which is even more weird. when i call client.close() client doesnt go anywhere. it is still there. browser is stillw ating for data. and when i kill connection in the browser, only then Tcp client gets closed and loop starts again.
namespace TestServer
{
class Program
{
public static void Main()
{
TcpListener server;
IPAddress addr = IPAddress.Parse("127.0.0.1");
server = new TcpListener(addr, 80);
server.Start();
byte[] buffer = new byte[1024];
while(true)
{
string data = null;
Console.WriteLine("Awaiting for connections");
TcpClient client = server.AcceptTcpClient();
Console.WriteLine("Connected...");
NetworkStream str = client.GetStream();
int msgCounter;
while ((msgCounter = str.Read(buffer, 0, buffer.Length)) != 0)
{
Console.WriteLine("Processing stream...");
data += System.Text.Encoding.ASCII.GetString(buffer, 0, msgCounter);
Console.WriteLine("Reciaved: {0}", System.Text.Encoding.ASCII.GetString(buffer, 0, msgCounter));
}
byte[] response = System.Text.Encoding.ASCII.GetBytes("200 OK");
str.Write(response, 0, response.Length);
str.Flush();
client.Close();
buffer = new byte[1024];
}
}
}
}

TCP as a bi-directional transport layer protocol does not denote any concept of the "client is done sending request" signal.
What it means for developers is that such signaling must be defined in the application (or any other higher level) protocol layer.
In your case it is declared by HTTP itself in the https://tools.ietf.org/html//rfc2616#section-5
So if you intend to implement an HTTP server you must parse the HTTP request that has a determined way to identify the end of the request (see the link above).
To summarise: you need to know somehow you've read the request entirely and you may start processing it and generating/sending the response.

I'd recommend you to start with a million times proven working MSDN example of TcpListener class. Additionally I can point to explicit wrong approach:
Do not recreate buffer, it is a waste of resources.
Do not use browser as a test client if you are working with TCP sockets. Any browser tries to correct somehow HTTP protocol errors and can do it in very unpredictable way. If you need HTTP level of debugging, use Fiddler, for lower levels - Microsoft Network Monitor, Wireshark of Netcat.
It could be useful to read some book about TCP/IP networking. Particularly, you will know, that there is not "close" operation or command for TCP connection by protocol definition, TcpClient just emulate it. Instead a peer can send "shutdown" to another one, it does mean it doesn't plan to send data anymore, but can read it. Connection can be considered as closed only after both peers have sent their "shutdown" and received "shutdown" from each other.

Weird tcp connection scenario

I am using TCP as a mechanism for keep alive here is my code:
Client
TcpClient keepAliveTcpClient = new TcpClient();
keepAliveTcpClient.Connect(HostId, tcpPort);
//this 'read' is supposed to blocked till a legal disconnect is requested
//or till the server unexpectedly dissapears
int numberOfByptes = keepAliveTcpClient.GetStream().Read(new byte[10], 0, 10);
//more client code...
Server
TcpListener _tcpListener = new TcpListener(IPAddress.Any, 1000);
_tcpListener.Start();
_tcpClient = _tcpListener.AcceptTcpClient();
Tracer.Write(Tracer.TraceLevel.INFO, "get a client");
buffer = new byte[10];
numOfBytes = _tcpClient.GetStream().Read(buffer, 0, buffer.Length);
if(numOfBytes==0)
{
//shouldn't reach here unless the connection is close...
}
I put only the relevant code... Now what that happens is that the client code is block on read as expected, but the server read return immediately with numOfBytes equals to 0, even if I retry to do read on the server it return immediately... but the client read is still block! so in the server side I think mistakenly that the client is disconnected from the server but the client thinks it connected to the server... someone can tell how it is possible? or what is wrong with my mechanism?
Edit: After a failure I wrote to the log these properties:
_tcpClient: _tcpClient.Connected=true
Socket: (_tcpClient.Client properties)
_tcpClient.Client.Available=0
_tcpClient.Client.Blocking=true
_tcpClient.Client.Connected=true
_tcpClient.Client.IsBound=true
Stream details
_tcpClient.GetStream().DataAvailable=false;

Even when correctly implemented, this approach will only detect some remote server failures. Consider the case where the intervening network partitions the two machines. Then, only when the underlying TCP stack sends a transport level keep-alive will the system detect the failure. Keepalive is a good description of the problem. [Does a TCP socket connection have a “keep alive”?] 2 is a companion question. The RFC indicates the functionality is optional.
The only certain way to reliably confirm that the other party is still alive is to occasionally send actual data between the two endpoints. This will result in TCP promptly detecting the failure and reporting it back to the application.

Maybe something that will give clue: it happens only when 10 or more clients
connect the server the same time(the server listen to 10 or more ports).
If you're writing this code on Windows 7/8, you may be running into a connection limit issue. Microsoft's license allows 20 concurrent connections, but the wording is very specific:
[Start->Run->winver, click "Microsoft Software License Terms"]
3e. Device Connections. You may allow up to 20 other devices to access software installed on the licensed computer to use only File Services, Print Services, Internet Information Services and Internet Connection Sharing and Telephony Services.
Since what you're doing isn't file, print, IIS, ICS, or telephony, it's possible that the previous connection limit of 10 from XP/Vista is still enforced in these circumstances. Set a limit of concurrent connections to 9 in your code temporarily, and see if it keeps happening.

The way I am interpretting the MSDN remarks it seems that behavior is expected. If you have no data the Read the method returns.
With that in mind I think what I would try is to send data at a specified interval like some of the previous suggestions along with a "timeout" of some sort. If you don't see the "ping" within your designated interval you could fail the keepalive. With TCP you have to keep in mind that there is no requirement to deem a connection "broken" just because you aren't seeing data. You could completely unplug the network cables and the connection will still be considered good up until the point that you send some data. Once you send data you'll see one of 2 behaviors. Either you'll never see a response (listening machine was shutdown?) or you'll get an "ack-reset" (listening machine is no longer listening on that particular socket)
https://msdn.microsoft.com/en-us/library/vstudio/system.net.sockets.networkstream.read(v=vs.100).aspx
Remarks:
This method reads data into the buffer parameter and returns the number of bytes successfully read. If no data is available for reading, the Read method returns 0. The Read operation reads as much data as is available, up to the number of bytes specified by the size parameter. If the remote host shuts down the connection, and all available data has been received, the Read method completes immediately and return zero bytes.

As I can see you are reading data on both sides, server and client. You need to write some data from the server to the client, to ensure that your client will have something to read. You can find a small test program below (The Task stuff is just to run the Server and Client in the same program).
class Program
{
private static Task _tcpServerTask;
private const int ServerPort = 1000;
static void Main(string[] args)
{
StartTcpServer();
KeepAlive();
Console.ReadKey();
}
private static void StartTcpServer()
{
_tcpServerTask = new Task(() =>
{
var tcpListener = new TcpListener(IPAddress.Any, ServerPort);
tcpListener.Start();
var tcpClient = tcpListener.AcceptTcpClient();
Console.WriteLine("Server got client ...");
using (var stream = tcpClient.GetStream())
{
const string message = "Stay alive!!!";
var arrayMessage = Encoding.UTF8.GetBytes(message);
stream.Write(arrayMessage, 0, arrayMessage.Length);
}
tcpListener.Stop();
});
_tcpServerTask.Start();
}
private static void KeepAlive()
{
var tcpClient = new TcpClient();
tcpClient.Connect("127.0.0.1", ServerPort);
using (var stream = tcpClient.GetStream())
{
var buffer = new byte[16];
while (stream.Read(buffer, 0, buffer.Length) != 0)
Console.WriteLine("Client received: {0} ", Encoding.UTF8.GetString(buffer));
}
}
}

Custom Http Listener in C#

The .NET has a built in HttpListener class, however I was wondering how I could roll my own Http Listener. I know there are a lot of implications to this, but I just want see the browser and my own app interact with each other.
This is the code I have written so far:
Socket servSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
servSocket.ExclusiveAddressUse = false; // Does this matter?
servSocket.Bind(new IPEndPoint(IPAddress.Any, 8080));
servSocket.Listen(10);
byte[] buffer;
do
{
try
{
Socket clientSocket = servSocket.Accept();
Console.WriteLine("Received Request at: {0}.", DateTime.Now);
EndPoint remoteEP = clientSocket.RemoteEndPoint;
buffer = new byte[1024];
clientSocket.ReceiveFrom(buffer, SocketFlags.None, ref remoteEP);
string request = Encoding.UTF8.GetString(buffer);
System.Diagnostics.Trace.Write(request);
buffer = Encoding.UTF8.GetBytes("<html><head><link rel=\"icon\" href=\"data:;base64,=\"></head><body></body></html>");
clientSocket.Send(buffer);
clientSocket.Close();
clientSocket.Dispose();
}
catch { }
}
while (this.isRunning);
This sort of works, however two issues I have noted, is that the clientSocket local end-point is on the same port as the servSocket. With the built-in HttpListener, the request gets handled by a random local end-point port. How can I mimic this?
I have set the ExclusiveAddressUse flag to false, however I still cannot bind more than one socket to that particular end-point, so what is its exact use?
Furthermore, from time to time I get a SocketException with 'An established connection was aborted by the software in your host machine'. What could be the source of the problem?

The simple answer is "you don't really care about that".
TCP works by estabilishing a two-way "virtual" persistent connection. To achieve this illusion, it uses a separate port for communication with a given client. You still connect to port 80 (for example), but a thousand clients use a thousand different ports for the actual communication with the server.
ExclusiveAddressUse allows you to bind more sockets to the same port - but each has to have its own IP address.
Networking is hard. Us as high an abstraction as you can, be it TcpListener, HttpListener or even OWIN.

Detect closed network connection

I've written a number of small programs that communicate via TCP. I'm having endless issues with the system hanging because one program has closed its network connection, and the other end-point somehow fails to notice that it's now disconnected.
I was expecting doing I/O on a TCP connection that has been closed to throw some kind of I/O exception, but instead the program seems to just hang, waiting forever for the other end-point to reply. Obviously if the connection is closed, that reply is never coming. (It doesn't even seem to time out if you leave it for, say, twenty minutes.)
Is there some way I can force the remote end to "see" that I've closed the network connection?
Update: Here is some code...
public sealed class Client
{
public void Connect(IPAddress target)
{
var socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
socket.Connect(ipAddress, 1177);
_stream = new NetworkStream(socket);
}
public void Disconnect()
{
_stream.Close();
}
}
public sealed class Server
{
public void Listen()
{
var listener = new TcpListener(IPAddress.Any, 1177);
listener.Start();
var socket = listener.AcceptSocket();
_stream = new NetworkStream(socket);
...
}
public void Disconnect()
{
socket.Shutdown(SocketShutdown.Both);
socket.Disconnect(false);
}
}

When an application closes a socket the right way, it sends a message containing 0 bytes. In some cases you may get a SocketException indicating something went wrong. In a third situation, the remote party is no longer connected (for instance by unplugging the network cable) without any communication between the two parties.
If that last thing happens, you'll have to write data to the socket in order to detect that you can no longer reach the remote party. This is why keep-alive mechanisms were invented - they check every so often whether they can still communicate with the other side.
Seeing the code you posted now: when using NetworkStream the Read operation on it would return a value of 0 (bytes) to indicate that the client has closed the connection.
The documentation is mentions both
"If no data is available for reading, the Read method returns 0."
and
"If the remote host shuts down the connection, and all available data has been received, the Read method completes immediately and return zero bytes."
in the same paragraph. In reality NetworkStream blocks if no data is available for reading while the connection is open.

Hi MathematicalOrchid,
You might find what you are looking for here:
http://blog.stephencleary.com/2009/05/detection-of-half-open-dropped.html
There is some great information there when it comes to working with TCP sockets and detecting half open connections.
You can also refer to this post which seems to have the same solution:
TcpClient communication with server to keep alive connection in c#?
-Dave

You are opening the socket, and assigning it to the stream. At the end of the process, you close the network stream, but not the socket.
For NetworkStream.Close() to close the underlying socket it must have the ownership parameters set to true in the constructor - See MSDN Docs at http://msdn.microsoft.com/en-us/library/te7e60bx.aspx.
This may result in the connection hanging as the underlying socket was not correctly closed.
Change
_stream = new NetworkStream(socket);
To
_stream = new NetworkStream(socket, true);
On a side note, if you do not require a maximum performance for your small app you should try using TCPClient instead - http://msdn.microsoft.com/en-us/library/system.net.sockets.tcpclient%28v=vs.100%29.aspx. This is a wrapper around socket and it provides connection state checking facilities.

How can I unbind a socket in C#?

I'm having some problems reusing a server socket in a test application I've made. Basically, I have a program that implements both the client side and the server side. I run two instances of this program for testing purposes, one instance starts to host and the other connects. This is the listening code:
private void Listen_Click(object sender, EventArgs e)
{
try
{
server = new ConnectionWrapper();
HideControls();
alreadyReset = false;
int port = int.Parse(PortHostEdit.Text);
IPEndPoint iep = new IPEndPoint(IPAddress.Any, port);
server.connection.Bind(iep); // bellow explanations refer to this line in particular
server.connection.Listen(1);
server.connection.BeginAccept(new AsyncCallback(OnClientConnected), null);
GameStatus.Text = "Waiting for connections on port " + port.ToString();
}
catch (Exception ex)
{
DispatchError(ex);
}
}
private void OnClientConnected(IAsyncResult iar)
{
try
{
me = Player.XPlayer;
myTurn = true;
server.connection = server.connection.EndAccept(iar); // I will only have one client, so I don't care for the original listening socket.
GameStatus.Text = server.connection.RemoteEndPoint.ToString() + " connected";
StartServerReceive();
}
catch (Exception ex)
{
DispatchError(ex);
}
}
This works fine the first time. However, after a while (when my little game ends), I call Dispose() on the server object, implemented like this:
public void Dispose()
{
connection.Close(); // connection is the actual socket
commandBuff.Clear(); // this is just a StringBuilder
}
I also have this in the object constructor:
public ConnectionWrapper()
{
commandBuff = new StringBuilder();
connection = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
connection.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
}
I get no error when I click the Listen button a second time. The client side connects just fine, however my server side does not detect the client connection a second time, which basically renders the server useless anyway. I'm guessing it's connecting to the old, lingering socket, but I have no idea why this is happening to be honest. Here's the client connection code:
private void Connect_Click(object sender, EventArgs e)
{
try
{
client = new ConnectionWrapper();
HideControls();
alreadyReset = false;
IPAddress ip = IPAddress.Parse(IPEdit.Text);
int port = int.Parse(PortConnEdit.Text);
IPEndPoint ipe = new IPEndPoint(ip, port);
client.connection.BeginConnect(ipe, new AsyncCallback(OnConnectedToServer), null);
}
catch (Exception ex)
{
DispatchError(ex);
}
}
If I do netstat -a in CMD, I see that the port I use is still bound and its state is LISTENING, even after calling Dispose(). I read that this is normal, and that there's a timeout for that port to be "unbound".
Is there a way I can force that port to unbind or set a very short timeout until it automatically gets unbound? Right now, it only gets unbound when I exit the program. Maybe I'm doing something wrong in my server? If so, what could that be? Why does the client connect fine, yet the server side doesn't detect it a second time?
I could make the socket always listen, not dispose it, and use a separate socket to handle the server connection, which would probably fix it, but I want other programs to be able to use the port between successive play sessions.
I remember seeing another question asking this, but there was no satisfactory answer for my case there.

There may be a couple of reasons why the port would stay open, but I think you should be able to resolve your issue by using an explicit LingerOption on the socket:
LingerOption lo = new LingerOption(false, 0);
socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.Linger, lo);
This basically turns the socket shutdown into an abortive shutdown instead of a graceful shutdown. If you want it to be graceful but just not wait as long, then use true in the constructor and specify a small but nonzero value for the timeout.
I just noticed this line, which is almost undoubtedly part of your problem:
server.connection = server.connection.EndAccept(iar); // I will only have one client, so I don't care for the original listening socket.
The comment you've written here is, well, wrong. Your wrapper class really shouldn't allow connection to be written to at all. But you cannot simply replace the listening socket with the client socket - they're two different sockets!
What's going to happen here is that (a) the listening socket goes out of scope and therefore never gets explicitly closed/disposed - this will happen at a random time, possibly at a nasty time. And (b) the socket that you do close is just the client socket, it will not close the listening socket, and so it's no wonder that you're having trouble rebinding another listening socket.
What you're actually witnessing isn't a socket timeout, it's the time it takes for the garbage collector to realize that the listening socket is dead and free/finalize it. To fix this, you need to stop overwriting the listening socket; the Dispose method of your wrapper class should dispose the original listening socket, and the client socket should be tracked separately and disposed whenever you are actually done with it.
In fact, you should really never need to rebind another listening socket at all. The listening socket stays alive the whole time. The actual connection is represented by just the client socket. You should only need to dispose the listening socket when you finally shut down the server.

I agree with the previous answer, you should also "shutdown" to allow any existing activity to complete and then close the socket flagging it for reuse...
socket.Shutdown(SocketShutdown.Both);
socket.Disconnect(true);