We Keep Coding

Httpwebrequest(client writemode) to TCPclient(Server Readmode) Approach

I am working on sending data across the network with the traditional server-client model.
Here Server starts the Tcplistener in a particular address and port. In this case, it is the local host.
The client makes use of WebRequest class in .net and takes the request stream and starts writing data into the request stream.
Let me walk through the Server code class:
TcpListener tcpListener = new TcpListener(IPAddress.Parse("127.0.0.1"),9309);
tcpListener.Start()//Start the listener and wait for client to connect.
while(true)
{
TcpClient newclient = tcpListener.AcceptTcpClient();
if(newclient.Connected())
{
break;
}
}
while(true)
{
ReadData(newclient);
}
public void ReadData(TcpClient newclient)
{
byte[] buffer = newbyte[50];
Stream ns = newclient.GetStream();
ns.Read(buffer, 0, buffer.Length);
Console.WriteLine( Encoding.UTF8.GetString(buffer));
}
//End of Server class.
Now let's see the Client Code class:-
WebRequest Request = HttpWebRequest.Create(http://127.0.0.1:9309/DataChannel);
Request.Method = "POST";
//Below method registers to Server's AcceptTcpClient and tcpclient is assigned.
Stream NetworkStream = ModifyCollimationRequest.GetRequestStream();
int DataWritten = 0;
while(true)
{
string Dname = "\r\nPosting server with Data as {0}\r\n";
byte[] dbytes = Encoding.UTF8.GetBytes(string.Format(Dname, ++DataWritten));
ns.WriteAsync(dbytes, 0, dbytes.Length);
ns.FlushAsync();
}
//End of Client code. Once the connection is established, the client keeps writing into the stream till the buffer size of >65000 is reached without issue but the problem is with the Server.
In Server,
Stream ns = newclient.GetStream(); -> This line under ReadData() method of the server executes,
but the the next line the code where Read() is used -> the code does not throw exception nor reaches next line. It just exits while debugging or times out. Someone it feels like, I am not able to fetch the stream or stream is empty. But the client keeps writing without any issue.
Can sometimes try this out and help me with what I am missing. Ultimately, I should be able to read the data available in a stream in any case but not sure why. Please add in your suggestions?

At the end of the client code, we need to call the below method for the data to
start actual streaming
ResponseStream = (HttpWebResponse)Request.GetResponse();
This is because there seems to be a bug in this .NET API where the buffered data is sent only when GetResponse() stream is called. This fixed the issue for me.
Below StackOverflow URL helped me learn more on this problem
HttpWebRequest.GetRequestStream returned stream does not send data immediately in .NET Core

C# TcpClient mistakenly reads the response of a previous Write

As a client, I have a queue of messages to send to the server. Each one opens a new TcpClient, Write and Read.
If the server has a delay, and the client reaches its timeout, I close the TcpClient and move to the next message.
However, when writing and reading the next one, I mistakenly read the previous response.
Here's a code that illustrates the problem:
using (TcpClient client1 = new TcpClient(ip, port))
using (NetworkStream stream1 = client1.GetStream())
{
byte[] dataToSend = Encoding.ASCII.GetBytes(message1);
await stream1.WriteAsync(dataToSend, 0, dataToSend.Length);
await stream1.FlushAsync();
// The server takes a while to respond. Simulate a timeout in client...
}
using (TcpClient client2 = new TcpClient(ip, port))
using (NetworkStream stream2 = client2.GetStream())
{
byte[] dataToSend = Encoding.ASCII.GetBytes(message2);
await stream2.WriteAsync(dataToSend, 0, dataToSend.Length);
await stream2.FlushAsync();
byte[] dataToRead = new byte[client2.ReceiveBufferSize];
int bytesRead = await stream2.ReadAsync(dataToRead, 0, client2.ReceiveBufferSize);
string messageReceived = Encoding.ASCII.GetString(dataToRead, 0, bytesRead);
}
The 'messageReceived' gets the response of the first Write rather than the second one.
I tried to set SendTimeout/ReceiveTimeout of the client, but it didn't help.
Is there an elegant way to ignore/avoid the response of the previous Write?

This is a problem with the server. Each connection is independent. A new connection will not see data for an old connection, even if the client and server are the same.
If a new connection is getting a response intended for a different connection, then that is a bug in the server. It sounds like the server may be sending responses to "whatever clients are currently connected", which is almost always incorrect behavior. Instead, the server should only send responses to the same connection that sent the request.

TcpClient stream write doesn't apply until stream/connection is closed

I'm trying to make a simple TCP program. I can get the client to connect to the server, but when my client calls stream.Write, the server does not read the sent data until the client closes the connection.
In addition to this, when my client attempts to read the server's response, the client throws an IOException. ("The connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.")
Server code:
listener.Start();
TcpClient client = listener.AcceptTcpClient();
NetworkStream ns = client.GetStream();
//Wait for message
MemoryStream ms = new MemoryStream();
ns.CopyTo(ms);//server will hang here until connection is closed, then receive correct data
Client code:
client.Connect(endpt);
client.GetStream().Write(encodedMessage, 0, encodedMessage.Length);
NetworkStream ns = client.GetStream();
MemoryStream ms = new MemoryStream();
ns.CopyTo(ms); //client crashes here

No, the server reads the data just fine. It's just that the stream doesn't end until you close the connection - the CopyTo method has no way to know that you want it to stop reading at some point. Remember, TCP doesn't send messages, it maintains a bi-directional stream of bytes. If you need messages, you need to build a messaging protocol on top of TCP.
The same thing happens on the client side, so you basically have a deadlock there - the client can't close the connection until it reads the whole stream, and the server can't close the connection until it reads the whole stream (indeed, it doesn't even accept any new connections, since you're doing all of this on one thread). But again, a TCP stream only ends when the connection is shutdown. So both are going to wait for each other forever.
I certainly don't get a crash, though. Are you trying to connect two clients to the same server at the same time? This will not work with your present code, and will cause a timeout like yours. The problem is that the connection is still waiting in the server's queue, and you don't call AcceptTcpClient until the previous connections are closed (again, the CopyTo deadlock).
Networking is quite hard. I'd certainly recommend you to use a well-tested, well-designed communication framework instead of rolling your own TCP-based protocol. Something like WCF or Lidgren will likely help quite a bit. If you really want to make your own TCP-based communication, you'll need to go through quite a bit of learning - I'd really like to recommend a good resource, but I haven't found any for C#/.NET so far. I've started a few samples on Networking, but that's far from production-ready code, and rather incomplete. It will show you the basic ideas of how an asynchronous TCP server is built in C#, as well as how to implement a simple HTTP-style or message-style TCP-based protocol.

You can try something like this:
//Server Part
public void Start()
{
Console.WriteLine("Server started...");
TcpListener listener = new TcpListener(System.Net.IPAddress.Loopback, 1234);
listener.Start();
while (true)
{
TcpClient client = listener.AcceptTcpClient();
new Thread(new ThreadStart(() =>
{
HandleClient(client);
})).Start();
}
}
private void HandleClient(TcpClient client)
{
NetworkStream stream = client.GetStream();
StreamWriter writer = new StreamWriter(stream, Encoding.ASCII) { AutoFlush = true };
StreamReader reader = new StreamReader(stream, Encoding.ASCII);
string inputLine = reader.ReadLine();
Console.WriteLine("The client with name " + " " + inputLine + " is conected");
}
//Client Part
public void InitClient()
{
client = new TcpClient("localhost", 1234);
stream = client.GetStream();
writer = new StreamWriter(stream) { AutoFlush = true };
reader = new StreamReader(stream);
}
public void SendMessage(string userName)
{
writer.WriteLine(userName);
}
//And here are the type of the variable that are used:
TcpClient client;
NetworkStream stream;
StreamWriter writer;
StreamReader reader;
Hope it help!

I resolved the issue by using AcceptSocket rather than AcceptTcpClient on the server, and using the Socket Receive function.
The client also now reads into an array rather than using the CopyTo function.
New server code:
listener.Start();
Socket socket = listener.AcceptSocket();
byte[] b = new byte[999];
socket.Receive(b);
New client code:
client.Connect(endpt);
client.GetStream().Write(encodedMessage, 0,encodedMessage.Length);
NetworkStream ns = client.GetStream();
byte[] bb = new byte[999];
ns.Read(bb, 0, 999);

How do I detect that my attempt to send data via TLS secured socket has failed?

This sample app creates a client-server connection via a TLS secured socket and sends some data over it:
static void Main(string[] args)
{
try
{
var listenerThread = new Thread(ListenerThreadEntry);
listenerThread.Start();
Thread.Sleep(TimeSpan.FromSeconds(1));
var socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.IP);
socket.Connect("localhost", Port);
var rawStream = new NetworkStream(socket);
using (var sslStream = new SslStream(rawStream, false, VerifyServerCertificate))
{
var certificate = new X509Certificate(CertsPath + #"test.cer");
var certificates = new X509CertificateCollection(new[] { certificate });
sslStream.AuthenticateAsClient("localhost", certificates, SslProtocols.Tls, false);
using (var writer = new StreamWriter(sslStream))
{
writer.WriteLine("TEST");
writer.Flush();
Thread.Sleep(TimeSpan.FromSeconds(10));
}
}
socket.Shutdown(SocketShutdown.Both);
socket.Disconnect(false);
Console.WriteLine("Success! Well, not really.");
}
catch (Exception exc)
{
Console.WriteLine(exc);
}
}
private static bool VerifyServerCertificate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
return true;
}
static void ListenerThreadEntry()
{
try
{
var listener = new TcpListener(IPAddress.Any, Port);
listener.Start();
var client = listener.AcceptTcpClient();
var serverCertificate = new X509Certificate2(CertsPath + #"\test.pfx");
var sslStream = new SslStream(client.GetStream(), false);
sslStream.AuthenticateAsServer(serverCertificate, false, SslProtocols.Tls, false);
client.Close(); // terminate the connection
using (var reader = new StreamReader(sslStream))
{
var line = reader.ReadLine();
Console.WriteLine("> " + line);
}
}
catch (Exception exc)
{
Console.WriteLine(exc);
}
}
The trick is that the connection is terminated from the server side immediately after handshake. And the problem is that the client side knows nothing about it; I'd expect the client side to raise an exception when it tries to send data over the closed connection, but it doesn't.
So, the question is: how do I detect such cases, when the connection was interrupted and data didn't really reach the server?

It is not possible to know which packets have arrived under the TCP model. TCP is a stream-oriented protocol, not a packet-oriented protocol; that is, it behaves like a bi-directional pipe. If you write 7 bytes, and then write 5 bytes, it's possible the other end will just get 12 bytes all at once. Worse, TCP's reliable delivery only guarantees that if the data arrives, it will do so in the correct order without duplication or rearrangement, and that if the data does not arrive, it will be resent.
If the connection is broken unexpectedly, TCP does not guarantee that you will know exactly what data was lost, nor is it reasonably possible to provide that information. The only thing the client knows is "I never received an acknowledgement for byte number N [and presumably not for the previous n bytes either], despite resending them multiple times." That is not enough information to determine whether byte N (and the other missing bytes) arrived at the server. It's possible that they did arrive and then the connection dropped, before the server could acknowledge them. It is also possible that they did not arrive at all. TCP cannot provide you with this information, because only the server knows it, and you are not connected to the server any longer.
Now, if you close the socket properly, using shutdown(2) or the .NET equivalent, then data in flight will be pushed through if possible, and the other end will error out promptly. Generally, we try to ensure that both sides agree on when to shutdown the connection. In HTTP, this is done with the Connection: Close header, in FTP with the BYE command, and so on. If one side shuts down unexpectedly, it may still cause data to be lost, because shutdown does not normally wait for acknowledgements.

Reading from closed NetworkStream doesn't cause any exception

I'm trying to create a rather simple client-server application, but for communication I want to use binary serialized objects. The communication itself seems rather fine, but when I close the stream on the client's side, the server doesn't really notice it and keeps on trying to read the stream.
server side (class Server, executed in separate thread):
listening for connections
listener = new TcpListener(IPAddress.Parse("127.0.0.1"), this.Port);
listener.Start();
while (!interrupted)
{
Console.WriteLine("Waiting for client");
TcpClient client = listener.AcceptTcpClient();
AddClient(client);
Console.WriteLine("Client connected");
}
adding the client:
public void AddClient(TcpClient socket)
{
Client client = new Client(this, socket);
this.clients.Add(client);
client.Start();
}
listening for messages (deep inside Client class):
BinaryFormatter deserializer = new BinaryFormatter();
while (!interrupted)
{
System.Diagnostics.Debug.WriteLine("Waiting for the message...");
AbstractMessage msg = (AbstractMessage)deserializer.Deserialize(stream);
System.Diagnostics.Debug.WriteLine("Message arrived: " + msg.GetType());
raiseEvent(msg);
}
unit test:
Server server = new Server(6666);
server.Start();
Thread.Sleep(500);
TcpClient client = new TcpClient("127.0.0.1", 6666);
var message = new IntroductionMessage();
byte[] arr = message.Serialize();
client.GetStream().Write(arr, 0, arr.Length);
Thread.Sleep(500);
Assert.AreEqual(1, server.Clients.Count);
client.GetStream().Close();
client.Close();
Thread.Sleep(1000);
Assert.AreEqual(0, server.Clients.Count);
server.Stop();
so the message gets read properly, but then, when I close the stream, deserializer.Deserialize(stream) doesn't appear to throw any exceptions... so should it just not be read this way, or should I close the client in a different way?

Assuming the stream used in the Server for deserializing the message is a NetworkStream (which is the type of the stream returned by TcpClient.GetStream()), you should do two things:
Define a specific message for "connection ends". When the server receives and deserializes this message, exit the while-loop. To make this work, the client obviously needs to send such a message before closing its TcpClient connection. (You might choose a different mechanism working in a similar manner -- but why not using the message mechanism you already have in place...)
Set a ReadTimeout on the NetworkStream, so in case the connection gets lost and the client is unable to send a "connection ends" message, the server will hit the timeout and realize that the client is "dead".
The code in your server for listening to client messages should look similar to this:
//
// Time-out after 1 minute after receiving last message
//
stream.ReadTimeOut = 60 * 1000;
BinaryFormatter deserializer = new BinaryFormatter();
try
{
while (!interrupted)
{
System.Diagnostics.Debug.WriteLine("Waiting for the message...");
AbstractMessage msg = (AbstractMessage)deserializer.Deserialize(stream);
System.Diagnostics.Debug.WriteLine("Message arrived: " + msg.GetType());
//
// Exit while-loop when receiving a "Connection ends" message.
// Adapt this if condition to whatever is appropriate for
// your AbstractMessage type.
//
if (msg == ConnectionEndsMessage) break;
raiseEvent(msg);
}
}
catch (IOException ex)
{
... handle timeout and other IOExceptions here...
}