We Keep Coding

Why does my C# TcpClient fail to receive content from the server if the server sends before reading everything from the client?

In an application I'm working on I want to disconnect clients that are trying to send me packets that are too large.
Just before disconnecting them I want to send them a message informing them about the reason for disconnecting them.
The issue I am running into is that the client cannot receive this server message, if the server does not read everything the client has send him first. I do not understand why this is happening.
I've managed to narrow it down to a very small test setup where the problem is demonstrated.
The StreamUtil class is a simple wrapper class that helps to get around the TCP message boundary problem, basically on the sender side it sends the size of each message first and then the message itself, and on the receiver side it receives the size of the message first and then the message.
The client uses a ReadKey command to simulate some time between sending and receiving, seeing in my real application these two actions are not immediately back to back either.
Here is a test case that works:
Run server as shown below
Run client as shown below, it will show a "Press key message", WAIT do not press key yet
Turn off server since everything is already in the clients receive buffer anyway (I validated this using packet sniffer)
Press key on the client -> client correctly shows the messages from the server.
This is what I was expecting, so great so far no problem yet.
Now in the server code, comment out the 2nd receive call and repeat the steps above.
Step 1 and 2 complete successfully, no errors sending from client to server.
On step 3 however the client crashes on the read from the server, EVEN though the server reply HAS arrived on the client (again validated with packet sniffer).
If I do a partial shutdown (eg socket.Shutdown (...send...)) without closing the socket on the server, everything works.
1: I just cannot get my head around WHY not processing the line of text from the client on the server causes the client to fail on receiving the text send back from the server.
2: If I send content from server to client but STOP the server before actually closing the socket, this content never arrives, but the bytes have already been transmitted to the server side... (see ReadKey in server to simulate, basically I block there and then just quit the server)
If anyone could shed light on these two issues, I'd deeply appreciate it.
Client:
class TcpClientDemo
{
public static void Main (string[] args)
{
Console.WriteLine ("Starting....");
TcpClient client = new TcpClient();
try
{
client.Connect("localhost", 56789);
NetworkStream stream = client.GetStream();
StreamUtil.SendString(stream, "Client teststring...");
Console.WriteLine("Press key to initiate receive...");
Console.ReadKey();
Console.WriteLine("server reply:" + StreamUtil.ReceiveString(stream));
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
finally
{
client.Close();
}
Console.WriteLine("Client ended");
Console.ReadKey(true);
}
}
Server:
class TcpServerDemo
{
public static void Main (string[] args)
{
TcpListener listener = new TcpListener (IPAddress.Any, 56789);
listener.Start ();
Console.WriteLine ("Waiting for clients to serve...");
while (true)
{
TcpClient client = null;
NetworkStream stream = null;
try
{
client = listener.AcceptTcpClient();
stream = client.GetStream();
//question 1: Why does commenting this line prevent the client from receiving the server reply??
Console.WriteLine("client string:" + StreamUtil.ReceiveString(stream));
StreamUtil.SendString(stream, "...Server reply goes here...");
//question 2: If I close the server program without actually calling client.Close (while on this line), the client program crashes as well, why?
//Console.ReadKey();
}
catch (Exception e)
{
Console.WriteLine(e.Message);
break;
}
finally
{
if (stream != null) stream.Close();
if (client != null) client.Close();
Console.WriteLine("Done serving this client, everything closed.");
}
}
listener.Stop();
Console.WriteLine("Server ended.");
Console.ReadKey(true);
}
}
StreamUtil:
public class StreamUtil
{
public static byte[] ReadBytes (NetworkStream pStream, int byteCount) {
byte[] bytes = new byte[byteCount];
int bytesRead = 0;
int totalBytesRead = 0;
try {
while (
totalBytesRead != byteCount &&
(bytesRead = pStream.Read (bytes, totalBytesRead, byteCount - totalBytesRead)) > 0
) {
totalBytesRead += bytesRead;
Console.WriteLine("Read/Total:" + bytesRead + "/" + totalBytesRead);
}
} catch (Exception e) {
Console.WriteLine(e.Message);
}
return (totalBytesRead == byteCount) ? bytes : null;
}
public static void SendString (NetworkStream pStream, string pMessage) {
byte[] sendPacket = Encoding.ASCII.GetBytes (pMessage);
pStream.Write (BitConverter.GetBytes (sendPacket.Length), 0, 4);
pStream.Write (sendPacket, 0, sendPacket.Length);
}
public static string ReceiveString (NetworkStream pStream) {
int byteCountToRead = BitConverter.ToInt32(ReadBytes (pStream, 4), 0);
Console.WriteLine("Byte count to read:"+byteCountToRead);
byte[] receivePacket = ReadBytes (pStream, byteCountToRead);
return Encoding.ASCII.GetString (receivePacket);
}
}

The client fails because it detects the socket was already closed.
If C# socket operations detect a closed connection during earlier operations, an exception is thrown on the next operation which can mask data which would otherwise have been received
The StreamUtil class does a couple of things when the connection is closed before/during a read:
Exceptions from the reads are swallowed
A read of zero bytes isn't treated
These obfuscate what's happening when an unexpected close hits the client.
Changing ReadBytes not to swallow exceptions and to throw a mock socket-closed exception (e.g. if (bytesRead == 0) throw new SocketException(10053);) when it reads zero bytes I think makes the outcome more clear.
Edit
I missed something subtle in your examples - your first example causes a TCP RST flag to be sent as soon as the server closes connection, due to the socket being closed with data waiting to be read.
The RST flag results in a closedown that doesn't preserve pending data.
This blog has some discussion based on a very similar scenario (web server sending a HTTP error).
So I don't think there's an easy fix, options are:
As you already tried, shutdown the socket on the server before closing to force a FIN to be sent before the RST
Read the data in question but never process it (taking up bandwidth for no reason)

Client Socket not getting reponse bytes in .NET

I am creating and sending a message on server socket and not getting any response in return , I would like to know what could be possible reasons for the same. Below is my sample code for client socket
static void Main()
{
int _DCPport = 9090;
IPHostEntry ipHostInfo = Dns.GetHostEntry("HostName");
Console.WriteLine("Got the IP Host Info" , ipHostInfo.ToString ());
IPAddress ipAddress = ipHostInfo.AddressList[0];
Console.WriteLine("Got the IP hadress Info");
//End point of the host where the socket will be connected
IPEndPoint remoteDAPEndPoint = new IPEndPoint(ipAddress, _DCPport);
// Create a clientSocket that connects to host
Socket clientDAP = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
clientDAP.Connect(remoteDAPEndPoint);
Console.WriteLine("Client socket connected to remote end point");
#region Input variables
string requestString = "Hello";
string responseString = "";
byte[] requestByte = System.Text.Encoding.UTF8.GetBytes(requestString);
byte[] responseByte = new byte[1024];
StringBuilder messageBuilder = new StringBuilder();
int byteResult = 0;
#endregion
try
{
//sending the string as bytes over the socket
Console.WriteLine("Sending the input " + requestString);
clientDAP.Send(requestByte);
}
catch (SocketException eX)
{
Console.WriteLine(eX.Message);
}
try
{
//recieving the response bytes
byteResult = clientDAP.Receive(responseByte, clientDAP.Available, SocketFlags.None);
Console.WriteLine("Recieved {0} bytes as response from remote end point" , byteResult);
if (clientDAP.Connected)
{
responseString = Encoding.UTF8.GetString(responseByte);
messageBuilder.Append(responseString);
}
}
catch (SocketException eX)
{
Console.WriteLine(eX.Message);
}
clientDAP.Shutdown(SocketShutdown.Both);
clientDAP.Disconnect(true);
string sResult = messageBuilder.ToString();
Console.WriteLine(sResult);
}
I am not getting any bytes when i call socket.Recieve() , Any reasons why such is happening ?

From Socket.Send:
There is also no guarantee that the data you send will appear on the network immediately. To increase network efficiency, the underlying system may delay transmission until a significant amount of outgoing data is collected. A successful completion of the Send method means that the underlying system has had room to buffer your data for a network send
Which means that the data may not have even arrived at the server yet when your call to Send has completed. Let alone that the server has had time to process your data and return a response.
And yet your code charges straight ahead to try to receive the server's response. This isn't unreasonable, in and of itself. Your call to Receive would have blocked until it had received some data if you had specified the size of your buffer, rather than using Available, which will happily return 0 if there's no data yet available. So your Receive call is asking for up to 0 bytes and that's how many you're being given.
So change this line
byteResult = clientDAP.Receive(responseByte, 1024, SocketFlags.None);
(Or better yet, introduce a constant and/or use responseByte.Length or switch to an overload where you don't explicitly specify how many bytes you want)

Reset TCP connection if server closes/crashes mid connection

I have a tcp connection like follows:
public void ConnectToServer()
{
string mac = GetUID();
while(true)
{
try
{
tcpClient = new TcpClient("xx.x.xx.xxx", xxxx);
networkstream = new SslStream(tcpClient.GetStream());
networkstream.AuthenticateAsClient("xx.x.xx.xxx");
networkstream.Write(Encoding.UTF8.GetBytes("0002:" + mac + "\r\n"));
networkstream.Flush();
string serverMessage = ReadMessage(networkstream);
Console.WriteLine("MESSAGE FROM SERVER: " + serverMessage);
}
catch (Exception e)
{
tcpClient.GetStream().Close();
tcpClient.Close();
}
}
}
This works fine and can send a receive data to/from the server.
What I need help with, if the server isn't running when the client starts, it'll wait and then connect once the server is up. But, if both the client and server are running and everything is working, if I close the server, the client will not reconnect(because I don't have anything to handle the event yet).
I have seen some answers on here that suggest polling and such. Is that the only way? The ReadMessage method that I call get into an infinite loop as well. I can post that code if need be.
I would really like to detect when the server closes/crashes and close the stream and the tcpclient and reconnect ASAP.
Here is my readmessage:
static string ReadMessage(SslStream sslStream)
{
if (sslStream.CanRead)
{
byte[] buffer = new byte[2048];
StringBuilder messageData = new StringBuilder();
int bytes = -1;
string message_type = null;
string actual_message = null;
do
{
try
{
Console.WriteLine("LENGTH: " + buffer.Length);
bytes = sslStream.Read(buffer, 0, buffer.Length);
Decoder decoder = Encoding.UTF8.GetDecoder();
char[] chars = new char[decoder.GetCharCount(buffer, 0, bytes)];
decoder.GetChars(buffer, 0, bytes, chars, 0);
messageData.Append(chars);
message_type = messageData.ToString().Substring(0, 5);
actual_message = messageData.ToString().Substring(5);
if (message_type.Equals("0001:"))
{
m_Window pop = new m_Window();
pop.callHttpPost(null, new EventArgs());
}
if (messageData.ToString().IndexOf("\r\n") != -1)
{
break;
}
}
catch (Exception e)
{
Console.WriteLine("ERROR: " + e.Message);
}
} while (bytes != 0);
return messageData.ToString();
}
return("CONNECTION HAS BEEN LOST");
}

With TCP you have 2 kinds of a server disconnect:
the server is closed
the server crashes
When the server is closed, you are going to receive 0 bytes on your client socket, this is the way you know that the peer has closed its end of the socket, which is called a half close.
But thing get more ugly if the server crashes.
When that happens again you have several possibilities.
If you don't send anything from the client to the server, the you have not way to find out that the server has indeed crashed.
The only way to find out that the server crashed is by letting the client send something or by activating keep alive. If you send something to a server socket that does not exist, you will have to wait a rather long period, because TCP is going to try several times, with retransmits untill there is a server response. When TCP has retried several times, then it will finally bail out and if you have a blocking socket you will see that the send failed, which means you should close your socket.
Actually there is a third possible server disconnect, that is a reset, but this is exceptionally used. I assume here that if there is a gracefull server shutdown, a normal close on the socket on the server end is executed. Which will end up in a FIN being sent instead of a RST, which is the exceptional case.
Now back to your situation, if the server crashes, it is inherently in the design of TCP, because of all those retransmission timeouts and increasing delays, that you will have to wait some time to actually detect that there is a problem. If the server is gracefully closed and startup again, this is not the case, this you detect immediately by receiving 0 bytes.

C# socket blocking with zero data

I have implemented C# tcp-ip client (both synchronous & async reading socket).
After each SocketException, I'm automatically reconnecting connection with server.
Then I have tested communication of client with ncat in windows. Here, if I kill ncat, it throws SocketException in C# client and everything works as I imagine.
But then I have tested it with ncat in linux - here communication works OK, but if I kill ncat server (the same settings like at Windows - ncat -l -k -p xxxx), huge amount of empty data (zero B) is received in callback (or waiting on socket in sync version) and no exception is thrown.
One thing is that Windows / Unix version of ncat can have different behavior. But still I need to solve this weird behavior for any version of tcp-ip server.
/* ------------------------------------------------------------------------- */
public void WaitForData()
{
try
{
if (callback == null)
callback = new AsyncCallback(OnDataReceived);
SocketPacket packet = new SocketPacket();
packet.thisSocket = socket;
m_result = socket.BeginReceive
(packet.dataBuffer, 0, 256,
SocketFlags.None, callback, packet);
}
catch (SocketException ex) { ///reconnecting }
}
/* ------------------------------------------------------------------------- */
public class SocketPacket
{
public System.Net.Sockets.Socket thisSocket;
public byte[] dataBuffer = new byte[256];
}
/* ------------------------------------------------------------------------- */
public void OnDataReceived(IAsyncResult asyn)
{
try
{
SocketPacket theSockId = (SocketPacket)asyn.AsyncState;
int iRx = theSockId.thisSocket.EndReceive(asyn);
char[] chars = new char[iRx];
System.Text.Decoder d = System.Text.Encoding.UTF8.GetDecoder();
int charLen = d.GetChars(theSockId.dataBuffer, 0, iRx, chars, 0);
string szData = new string(chars);
szData = szData.Replace("\n", String.Empty);
processMessage(szData);
WaitForData();
}
catch (ObjectDisposedException) { }
catch (SocketException ex) { ///reconnecting }
}
Thank you!

Solved
In OnDataReceived callback, I check amount of incoming data, so I do:
if (iRx == 0)
throw new SocketException(Convert.ToInt16(SocketError.HostDown));

TCP receiver can never get the exact status of the remote connection. Killing the process will send FIN (receive is zero) or RST(get exception) depending on the TCP/IP stack implementation. There are other condition where the application are completely unaware of the connection breakage. Suppose if the remote system is forcefully reset or the cable wire is unplugged or the IP address is changed, there is no way you can get to know the connection broken until you send some data. If your application need to know the status of the connection, it can send some dummy data every 1 minute to make sure the connection alive.

TcpClient - An existing connection was forcibly closed by the remote host

The Info
I have been developing a web http server in c# and decided to add a remote console feature. The console can be used from any location and uses a TcpListener (web server) and a TcpClient (remote console) to send commands and functions through.
The Code
This is what my server looks like:
TcpListener consoleListener = new TcpListener(consolePort);
consoleListener.Start();
byte[] bytes = new Byte[256];
string data = null;
while (true)
{
TcpClient client = consoleListener.AcceptTcpClient();
data = null;
byte[] msg = { 0 };
int i;
while ((i = client.GetStream().Read(bytes, 0, bytes.Length)) != 0)
{
data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
if (data == "shutdown")
{
//Server shutdown logic.
}
//Other commands here...
else
{
msg = Encoding.ASCII.GetBytes("Invalid command. Type 'help' or '?' to get a list of commands.");
}
client.GetStream().Write(msg, 0, msg.Length); //sends return message to console
}
client.Close(); //closes connection between client and server after EVERY command. Connection is reopened when a new command is sent.
}
Note - The server is run on a separate thread to both the webserver and main console application thread.
This is my client:
public static string Communicate(string text)
{
try
{
TcpClient client = new TcpClient(ip, port); //initializes tcpclient (ip and port are correct)
byte[] data = System.Text.Encoding.ASCII.GetBytes(text); //converts text to bytes for stream writing
NetworkStream stream = client.GetStream();
stream.Write(data, 0, data.Length);
Console.WriteLine("Sent data: " + text);
data = new Byte[256];
string responseData = String.Empty; //initializes responsData string
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
client.Close();
return responseData; //returns what server writes
}
catch (Exception ex)
{
return "An error occured\n" + ex.ToString();
}
}
The Problem
I can send one command to the server with a successful return. However, when I try and send another command, the server throws the error below:
System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
--- End of inner exception stack trace ---
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at ---.Server.ConsoleListener() in X:\Users\---\Documents\Visual Studio 2013\Projects\---\---\Program.cs:line x
I know it is not firewall or administrator elevation problems as I can send one command through successfully. It is only on the second command sent that it throws this error.
Here is a screenshot describing the problem:
EDIT: By doing a little research, I found that the problem is most likely a result of a small error in my for loop. However, I do not know any way of fixing this as I do not know the exact problem :). Please help me identify it so I can fix it.
Thanks again

It seems that your client closes the connection after one message.
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
client.Close();
return responseData; //returns what server writes
If you want to persist the connection you should have a loop on the client similar to the one you have on the server. If you want to establish a new connection every time you should close the stream gracefully on the server and not have a loop like this. You will still need to loop in case the message is longer or you need to specify max length for the command.

I don't know if you fixed your issue or not but I guess you should post your workaround at least so others can check it.
I don't fully understand your issue but I had the same exception, but mine was triggered while the client disconnected and server was trying to read the stream (networkStream).
I had a single command for reading
networkstream.Read(mybuffer, 0, mybuffer.length);
As the checked answer suggested I changed that for:
do
{
byte[] buff = new byte[1];
networkstream.Read(buff, 0, 1);
myreceivedbuff.Add(buff);
} while (networkstream.DataAvailable)
this also produced the issue while client disc, so I had to do this
do
{
byte[] buff = new byte[1];
try
{
networkstream.Read(buff, 0, 1);
}
catch(exception ex)
{
throw new exception("The dam client disconnected in the middle of a transaction.");
}
myreceivedbuff.Add(buff);
} while (networksteam.DataAvailable)
I had to do this since it doesn't matter if its on a client or a server the exception is the same. host disconnection meanwhile my exception was CLIENT disconnection and this generic message misguided me to the solution.
Sorry if the code is not pasted from vs but I typed here so fix the capitalization so it can compile.
Hope this helps someone.

I had same solution. it is usually happens if client is disconnected. Solution from Alex RG is not working unfortunately. you will get another exception. Best solutions is described here by Microsoft
you need to check using CanRead
TcpClient tcpClient = new TcpClient ();
// Uses the GetStream public method to return the NetworkStream.
NetworkStream netStream = tcpClient.GetStream ();
if (netStream.CanRead)
{
// Reads NetworkStream into a byte buffer.
byte[] bytes = new byte[tcpClient.ReceiveBufferSize];
// Read can return anything from 0 to numBytesToRead.
// This method blocks until at least one byte is read.
netStream.Read (bytes, 0, (int)tcpClient.ReceiveBufferSize);
// Returns the data received from the host to the console.
string returndata = Encoding.UTF8.GetString (bytes);
Console.WriteLine ("This is what the host returned to you: " + returndata);
}
else
{
Console.WriteLine ("You cannot read data from this stream.");
tcpClient.Close ();
// Closing the tcpClient instance does not close the network stream.
netStream.Close ();
return;
}