We Keep Coding

C# TcpClient mistakenly reads the response of a previous Write

As a client, I have a queue of messages to send to the server. Each one opens a new TcpClient, Write and Read.
If the server has a delay, and the client reaches its timeout, I close the TcpClient and move to the next message.
However, when writing and reading the next one, I mistakenly read the previous response.
Here's a code that illustrates the problem:
using (TcpClient client1 = new TcpClient(ip, port))
using (NetworkStream stream1 = client1.GetStream())
{
byte[] dataToSend = Encoding.ASCII.GetBytes(message1);
await stream1.WriteAsync(dataToSend, 0, dataToSend.Length);
await stream1.FlushAsync();
// The server takes a while to respond. Simulate a timeout in client...
}
using (TcpClient client2 = new TcpClient(ip, port))
using (NetworkStream stream2 = client2.GetStream())
{
byte[] dataToSend = Encoding.ASCII.GetBytes(message2);
await stream2.WriteAsync(dataToSend, 0, dataToSend.Length);
await stream2.FlushAsync();
byte[] dataToRead = new byte[client2.ReceiveBufferSize];
int bytesRead = await stream2.ReadAsync(dataToRead, 0, client2.ReceiveBufferSize);
string messageReceived = Encoding.ASCII.GetString(dataToRead, 0, bytesRead);
}
The 'messageReceived' gets the response of the first Write rather than the second one.
I tried to set SendTimeout/ReceiveTimeout of the client, but it didn't help.
Is there an elegant way to ignore/avoid the response of the previous Write?

This is a problem with the server. Each connection is independent. A new connection will not see data for an old connection, even if the client and server are the same.
If a new connection is getting a response intended for a different connection, then that is a bug in the server. It sounds like the server may be sending responses to "whatever clients are currently connected", which is almost always incorrect behavior. Instead, the server should only send responses to the same connection that sent the request.

TcpClient stream write doesn't apply until stream/connection is closed

I'm trying to make a simple TCP program. I can get the client to connect to the server, but when my client calls stream.Write, the server does not read the sent data until the client closes the connection.
In addition to this, when my client attempts to read the server's response, the client throws an IOException. ("The connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.")
Server code:
listener.Start();
TcpClient client = listener.AcceptTcpClient();
NetworkStream ns = client.GetStream();
//Wait for message
MemoryStream ms = new MemoryStream();
ns.CopyTo(ms);//server will hang here until connection is closed, then receive correct data
Client code:
client.Connect(endpt);
client.GetStream().Write(encodedMessage, 0, encodedMessage.Length);
NetworkStream ns = client.GetStream();
MemoryStream ms = new MemoryStream();
ns.CopyTo(ms); //client crashes here

No, the server reads the data just fine. It's just that the stream doesn't end until you close the connection - the CopyTo method has no way to know that you want it to stop reading at some point. Remember, TCP doesn't send messages, it maintains a bi-directional stream of bytes. If you need messages, you need to build a messaging protocol on top of TCP.
The same thing happens on the client side, so you basically have a deadlock there - the client can't close the connection until it reads the whole stream, and the server can't close the connection until it reads the whole stream (indeed, it doesn't even accept any new connections, since you're doing all of this on one thread). But again, a TCP stream only ends when the connection is shutdown. So both are going to wait for each other forever.
I certainly don't get a crash, though. Are you trying to connect two clients to the same server at the same time? This will not work with your present code, and will cause a timeout like yours. The problem is that the connection is still waiting in the server's queue, and you don't call AcceptTcpClient until the previous connections are closed (again, the CopyTo deadlock).
Networking is quite hard. I'd certainly recommend you to use a well-tested, well-designed communication framework instead of rolling your own TCP-based protocol. Something like WCF or Lidgren will likely help quite a bit. If you really want to make your own TCP-based communication, you'll need to go through quite a bit of learning - I'd really like to recommend a good resource, but I haven't found any for C#/.NET so far. I've started a few samples on Networking, but that's far from production-ready code, and rather incomplete. It will show you the basic ideas of how an asynchronous TCP server is built in C#, as well as how to implement a simple HTTP-style or message-style TCP-based protocol.

You can try something like this:
//Server Part
public void Start()
{
Console.WriteLine("Server started...");
TcpListener listener = new TcpListener(System.Net.IPAddress.Loopback, 1234);
listener.Start();
while (true)
{
TcpClient client = listener.AcceptTcpClient();
new Thread(new ThreadStart(() =>
{
HandleClient(client);
})).Start();
}
}
private void HandleClient(TcpClient client)
{
NetworkStream stream = client.GetStream();
StreamWriter writer = new StreamWriter(stream, Encoding.ASCII) { AutoFlush = true };
StreamReader reader = new StreamReader(stream, Encoding.ASCII);
string inputLine = reader.ReadLine();
Console.WriteLine("The client with name " + " " + inputLine + " is conected");
}
//Client Part
public void InitClient()
{
client = new TcpClient("localhost", 1234);
stream = client.GetStream();
writer = new StreamWriter(stream) { AutoFlush = true };
reader = new StreamReader(stream);
}
public void SendMessage(string userName)
{
writer.WriteLine(userName);
}
//And here are the type of the variable that are used:
TcpClient client;
NetworkStream stream;
StreamWriter writer;
StreamReader reader;
Hope it help!

I resolved the issue by using AcceptSocket rather than AcceptTcpClient on the server, and using the Socket Receive function.
The client also now reads into an array rather than using the CopyTo function.
New server code:
listener.Start();
Socket socket = listener.AcceptSocket();
byte[] b = new byte[999];
socket.Receive(b);
New client code:
client.Connect(endpt);
client.GetStream().Write(encodedMessage, 0,encodedMessage.Length);
NetworkStream ns = client.GetStream();
byte[] bb = new byte[999];
ns.Read(bb, 0, 999);

Why does my C# TcpClient fail to receive content from the server if the server sends before reading everything from the client?

In an application I'm working on I want to disconnect clients that are trying to send me packets that are too large.
Just before disconnecting them I want to send them a message informing them about the reason for disconnecting them.
The issue I am running into is that the client cannot receive this server message, if the server does not read everything the client has send him first. I do not understand why this is happening.
I've managed to narrow it down to a very small test setup where the problem is demonstrated.
The StreamUtil class is a simple wrapper class that helps to get around the TCP message boundary problem, basically on the sender side it sends the size of each message first and then the message itself, and on the receiver side it receives the size of the message first and then the message.
The client uses a ReadKey command to simulate some time between sending and receiving, seeing in my real application these two actions are not immediately back to back either.
Here is a test case that works:
Run server as shown below
Run client as shown below, it will show a "Press key message", WAIT do not press key yet
Turn off server since everything is already in the clients receive buffer anyway (I validated this using packet sniffer)
Press key on the client -> client correctly shows the messages from the server.
This is what I was expecting, so great so far no problem yet.
Now in the server code, comment out the 2nd receive call and repeat the steps above.
Step 1 and 2 complete successfully, no errors sending from client to server.
On step 3 however the client crashes on the read from the server, EVEN though the server reply HAS arrived on the client (again validated with packet sniffer).
If I do a partial shutdown (eg socket.Shutdown (...send...)) without closing the socket on the server, everything works.
1: I just cannot get my head around WHY not processing the line of text from the client on the server causes the client to fail on receiving the text send back from the server.
2: If I send content from server to client but STOP the server before actually closing the socket, this content never arrives, but the bytes have already been transmitted to the server side... (see ReadKey in server to simulate, basically I block there and then just quit the server)
If anyone could shed light on these two issues, I'd deeply appreciate it.
Client:
class TcpClientDemo
{
public static void Main (string[] args)
{
Console.WriteLine ("Starting....");
TcpClient client = new TcpClient();
try
{
client.Connect("localhost", 56789);
NetworkStream stream = client.GetStream();
StreamUtil.SendString(stream, "Client teststring...");
Console.WriteLine("Press key to initiate receive...");
Console.ReadKey();
Console.WriteLine("server reply:" + StreamUtil.ReceiveString(stream));
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
finally
{
client.Close();
}
Console.WriteLine("Client ended");
Console.ReadKey(true);
}
}
Server:
class TcpServerDemo
{
public static void Main (string[] args)
{
TcpListener listener = new TcpListener (IPAddress.Any, 56789);
listener.Start ();
Console.WriteLine ("Waiting for clients to serve...");
while (true)
{
TcpClient client = null;
NetworkStream stream = null;
try
{
client = listener.AcceptTcpClient();
stream = client.GetStream();
//question 1: Why does commenting this line prevent the client from receiving the server reply??
Console.WriteLine("client string:" + StreamUtil.ReceiveString(stream));
StreamUtil.SendString(stream, "...Server reply goes here...");
//question 2: If I close the server program without actually calling client.Close (while on this line), the client program crashes as well, why?
//Console.ReadKey();
}
catch (Exception e)
{
Console.WriteLine(e.Message);
break;
}
finally
{
if (stream != null) stream.Close();
if (client != null) client.Close();
Console.WriteLine("Done serving this client, everything closed.");
}
}
listener.Stop();
Console.WriteLine("Server ended.");
Console.ReadKey(true);
}
}
StreamUtil:
public class StreamUtil
{
public static byte[] ReadBytes (NetworkStream pStream, int byteCount) {
byte[] bytes = new byte[byteCount];
int bytesRead = 0;
int totalBytesRead = 0;
try {
while (
totalBytesRead != byteCount &&
(bytesRead = pStream.Read (bytes, totalBytesRead, byteCount - totalBytesRead)) > 0
) {
totalBytesRead += bytesRead;
Console.WriteLine("Read/Total:" + bytesRead + "/" + totalBytesRead);
}
} catch (Exception e) {
Console.WriteLine(e.Message);
}
return (totalBytesRead == byteCount) ? bytes : null;
}
public static void SendString (NetworkStream pStream, string pMessage) {
byte[] sendPacket = Encoding.ASCII.GetBytes (pMessage);
pStream.Write (BitConverter.GetBytes (sendPacket.Length), 0, 4);
pStream.Write (sendPacket, 0, sendPacket.Length);
}
public static string ReceiveString (NetworkStream pStream) {
int byteCountToRead = BitConverter.ToInt32(ReadBytes (pStream, 4), 0);
Console.WriteLine("Byte count to read:"+byteCountToRead);
byte[] receivePacket = ReadBytes (pStream, byteCountToRead);
return Encoding.ASCII.GetString (receivePacket);
}
}

The client fails because it detects the socket was already closed.
If C# socket operations detect a closed connection during earlier operations, an exception is thrown on the next operation which can mask data which would otherwise have been received
The StreamUtil class does a couple of things when the connection is closed before/during a read:
Exceptions from the reads are swallowed
A read of zero bytes isn't treated
These obfuscate what's happening when an unexpected close hits the client.
Changing ReadBytes not to swallow exceptions and to throw a mock socket-closed exception (e.g. if (bytesRead == 0) throw new SocketException(10053);) when it reads zero bytes I think makes the outcome more clear.
Edit
I missed something subtle in your examples - your first example causes a TCP RST flag to be sent as soon as the server closes connection, due to the socket being closed with data waiting to be read.
The RST flag results in a closedown that doesn't preserve pending data.
This blog has some discussion based on a very similar scenario (web server sending a HTTP error).
So I don't think there's an easy fix, options are:
As you already tried, shutdown the socket on the server before closing to force a FIN to be sent before the RST
Read the data in question but never process it (taking up bandwidth for no reason)

C# Client/Server returning answer from server to client

I've created a simple client/server program which takes an input from the client and returns a answer to the input by looking in the text file to see if there is an answer associated with the input.
The issue I'm having is that I get the response on the server side but I don't know how to send it back to the client (it just returns the input on the client side).
The second issue is that it will execute once, as in, it will only take in one input. I tried adding a loop but couldn't get it to work.
Server.cs
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Net.Sockets;
using System.Net;
namespace server
{
class server
{
const string SERVER_IP = "127.0.0.1";
static void Main(string[] args)
{
//Create Dictionary
Dictionary<string, string> dict = new Dictionary<string, string>();
//---Read Text File containing commands ---
StreamReader sr = new StreamReader(#"C:\Users\Desktop\potato.txt");
string line;
//Splits the text into commands:responses
while ((line = sr.ReadLine()) != null)
{
string[] arr = line.Split(';');
dict.Add(arr[0], arr[1]);
}
//Print dictionary TESTING FUNCTION
foreach (KeyValuePair<string, string> kvp in dict)
{
Console.WriteLine("Command = {0} Response = {1}", kvp.Key, kvp.Value);
}
//---Input the port number for clients to conect---
Console.Write("Input port" + System.Environment.NewLine);
int PORT_NO = int.Parse(Console.ReadLine());
//---listen at the specified IP and port no.---
IPAddress localAdd = IPAddress.Parse(SERVER_IP);
TcpListener listener = new TcpListener(localAdd, PORT_NO);
Console.WriteLine("Listening for Commands");
listener.Start();
//---incoming client connected---
TcpClient client = listener.AcceptTcpClient();
//---get the incoming data through a network stream---
NetworkStream nwStream = client.GetStream();
byte[] buffer = new byte[client.ReceiveBufferSize];
//---read incoming stream---
int bytesRead = nwStream.Read(buffer, 0, client.ReceiveBufferSize);
//---convert the command data received into a string---
string dataReceived = Encoding.ASCII.GetString(buffer, 0, bytesRead);
Console.WriteLine("Received Command : " + dataReceived);
//---Search Command and send a response
string Response;
if (dict.TryGetValue(dataReceived, out Response))
{
Console.WriteLine(Response);
}
//---write back the response to the client---
Console.WriteLine("Sending Response : " + Response);
nwStream.Write(buffer, 0, bytesRead);
Console.ReadLine();
}
}
}

You need to convert Response to a byte[] just as you do in the client sending your request (i.e. bytesToSend). E.g.:
Console.WriteLine("Sending Response : " + Response);
byte[] bytesToSend = ASCIIEncoding.ASCII.GetBytes(Response);
nwStream.Write(bytesToSend, 0, bytesToSend.Length);
Console.ReadLine();
That said, you have made the classic mistake every person does who tries to write TCP code without first reading references about TCP and sockets: you mistakenly believe that when you read from a socket, you will always receive in that single operation every byte that was read. So even with the above fix (which does address the issue on the server side), it is possible you will see partial responses on the client side (not as likely when testing locally, but much more likely if you move to running on the Internet or across LANs, and especially as the message size increases).
For low-volume network interaction, you may want to consider wrapping your NetworkStream with StreamReader and StreamWriter objects, and use ReadLine() and WriteLine() to receive and send data (i.e. use line-breaks as the delimiter for the data).
As for dealing with multiple requests, given the code you have presented here, the simplest approach is to add a loop around the server code after the listener.Start() method. I.e. containing all the code after that statement, starting with the call to listener.AcceptTcpClient() and going to the last statement in the method. However, again this is only appropriate for low-volume network code. If you anticipate clients will need your server to handle multiple requests and especially if in quick succession, what you really want is for the client to maintain the connection, i.e. only connect once and then have it send multiple requests on that same connection.
Similarly, if you want to be able to handle multiple clients at once, you cannot run the server in a single thread. Instead, at the very least you'll need to use the thread-blocking logic you're using now, where you have a new thread created for each client. Better, would be to use the non-blocking API (e.g. NetworkStream.BeginRead(), StreamReader.ReadLineAsync(), etc. … there are many asynchronous options to choose from), and let .NET deal with the threading for you.
Doing it that way will require significant changes to the server code. You really should look carefully at various samples on MSDN and Stack Overflow to see how this sort of thing is done. I also strongly recommend you read through the Winsock Programmer's FAQ. It is not specifically about .NET at all, but does cover all of the key details you'll need to know in order to effectively and correctly use the .NET API.

Half-closing TcpClient

I'm having serious issue with half-closing a TcpClient. What I am trying to do is:
On the client:
Send a message
Shutdown the underlying socket for sending
Receive a response
Shutdown the underlying socket for reading (or, at this point, just close it)
On the server:
Receive a message
Shutdown the underlying socket for reading
Send a response
Shutdown the underlying socket for writing (or, at this point, just close it)
However, after the step 2 on either the client, or the server, I can't use the TcpClient's stream.
Here's a very simplified version of my code (without asynchronous calls, processing and cleaning up, and also using StreamReader and StreamWriter instead of an XmlSerializer):
//initialize the connection between the server and the client
var listener = new TcpListener(IPAddress.Any, 13546);
listener.Start();
var client = new TcpClient("127.0.0.1", 13546);
var server = listener.AcceptTcpClient();
listener.Stop();
//CLIENT: send the message
var cwriter = new StreamWriter(client.GetStream());
cwriter.Write("client's message");
cwriter.Flush();
client.Client.Shutdown(SocketShutdown.Send);
//SERVER: receive the message
string msg;
var sreader = new StreamReader(server.GetStream());
msg = sreader.ReadToEnd();
server.Client.Shutdown(SocketShutdown.Receive);
//SERVER: send a response
//Here the code fails on server.GetStream() -
//InvalidOperationException, apparently the whole connection is closed now
var swriter = new StreamWriter(server.GetStream());
swriter.Write(msg + " with server's response");
swriter.Flush();
server.Client.Shutdown(SocketShutdown.Send);
//CLIENT: receive the message
var creader = new StreamReader(client.GetStream());
var response = creader.ReadToEnd();
client.Client.Shutdown(SocketShutdown.Receive);
Is there any way to do this without using a raw socket? Is there something I'm getting wrong?

The problem is that ReadToEnd reads data up to the end of stream. By issuing Client.Shutdown you actually close the socket making it impossible to reuse it (at least in case of TCPClient). Here's the code of GetStream()
public NetworkStream GetStream() {
if(Logging.On)Logging.Enter(Logging.Sockets, this, "GetStream", "");
if (m_CleanedUp){
throw new ObjectDisposedException(this.GetType().FullName);
}
if (!Client.Connected) {
throw new InvalidOperationException(SR.GetString(SR.net_notconnected));
}
if (m_DataStream==null) {
m_DataStream = new NetworkStream(Client, true);
}
if(Logging.On)Logging.Exit(Logging.Sockets, this, "GetStream", m_DataStream);
return m_DataStream;
}
As you can see, the error occures due to closed socket.
EDIT: It is ridiculously strange but I think I found the reason why it doesn't work properly. The reason is that Shutdown always sets flags for the entire socket as disconnected. Even though we are actually not closing it that way! If we preserve stream at the begining of the method we will not face this problem since the problem lies in GetStream method which checks socket's state. But we are probably exposed to some bugs when other code would check socket's state.