We Keep Coding

How to set a correct buffer size for TCP message responses

This is how I currently send data to an external TCP server
byte[] data = new byte[0] /* the data to send */;
TcpClient client = new TcpClient("127.0.0.1", 3000); // connect to the tcp server
NetworkStream stream = client.GetStream();
await stream.WriteAsync(data, 0, data.Length);
data = new byte[256]; // set the buffer size
int responseBytes = await stream.ReadAsync(data, 0, data.Length); // store the response to the buffer
string responseData = System.Text.Encoding.ASCII.GetString(data, 0, responseBytes);
stream.Close();
client.Close();
For the response I have to setup the buffer size here new byte[256]. But what if the response is greater than this size? I can't determine the correct size because I'm just connecting to his external server, send a message to it and expect a response. Is there a way I can make this dynamic?
As a sidenote: I'm sending various HL7 messages to clinic servers and they will send back HL7 ACK messages as a response. This gives some information about HL7 ACK messages
https://healthstandards.com/blog/2007/02/01/ack-message-original-mode-acknowledgement/
An example ACK could be
MSH|^~&|CATH|StJohn|AcmeHIS|StJohn|20061019172719||ACK^O01|MSGID12349876|P|2.3
MSA|AA|MSGID12349876

For the response I have to setup the buffer size here new byte[256]. But what if the response is greater than this size?
Then you call stream.ReadAsync() and append your buffer (or the decoded string) to a larger buffer until you know you have received the entire message, which you need to do anyway: the Write() from one end of the socket does not need to correspond to one Read() on the other end. Multiple writes can be read in a single read, or the other way around.
So something like this:
data = new byte[256]; // set the buffer size
var builder = new StringBuilder();
do
{
int responseBytes = await stream.ReadAsync(data, 0, data.Length); // store the response to the buffer
string responseData = System.Text.Encoding.ASCII.GetString(data, 0, responseBytes);
builder.Append(responseData);
} while (responseBytes > 0)
Do note that this happens to work with ASCII, as it doesn't have multibyte characters. Were it UTF-8 or a similar encoding, the 256th byte could be the start of a character which continues into the next read, i.e. byte 1 (and perhaps 2) of the next read.
This code also assumes you want to keep reading until the connection is closed (then responseBytes = 0). If this protocol has a length prefix or message terminator, you have to handle those.
Usually you don't want to implement this low-level stuff yourself, aren't there libraries available that handle the HL7 protocol?

What is the correct method to read from a NetworkStream?

I'm developing a software that will constantly receive requests of various types and different lengths.
In particular, the length is the part that interest me because I don't know the length of the message that I will go to receive.
For the moment I wrote this code that goes to write the received bytes in a byte[] of a very large size, and then copies in a second byte[] only the length of the data.
//get the network stream
NetworkStream networkStream = tcpClient.GetStream();
//initialize an bytes array of size 5 MB
byte[] largeArray = new byte[5242880];
//write the bytes in the largeArray
//determines the length of the received message
int lenght = networkStream.Read(largeArray, 0, largeArray.Length);
//initialize an bytes array with the correct message length
byte[] messageArray = new byte[lenght];
//make a copy from the largeArray to the messageArray with the right lenght
Buffer.BlockCopy(largeArray, 0, messageArray, 0, messageArray.Length);
I wanted to know if this method can be corrected or are there better alternatives to receive messages of unknown lengths ?

C# TCP NetworkStream missing a few bytes of data

I'm having two problems and after trying a few techniques I've read on stackoverflow, the problem persists. I'm trying to send a file from the server to client with the following code below but the problem is that the file is always a few bytes short, causing file corruption.. The second problem is that the stream doesn't close despite implementing a zero length packet at the end to indicate the transfer is finished without closing the connection.
Server code snippet:
/*
* Received request from client for file, sending file to client.
*/
//open file to send to client
FileStream fs = new FileStream(fileLocation, FileMode.Open, FileAccess.Read);
byte[] data = new byte[1024];
long fileSize = fs.Length;
long sent = 0;
int count = 0;
while (sent < fileSize)
{
count = fs.Read(data, 0, data.Length);
netStream.Write(data, 0, count);
sent += count;
}
netStream.Write(new byte[1024], 0, 0); //send zero length byte stream to indicate end of file.
fs.Flush();
netStream.Flush();
Client code snippet:
TcpClient client;
NetworkStream serverStream;
/*
* [...] client connect
*/
//send request to server for file
byte[] dataToSend = SerializeObject(obj);
serverStream.Write(dataToSend, 0, dataToSend.Length);
//create filestream to save file
FileStream fs = new FileStream(fileName, FileMode.Create, FileAccess.Write);
//handle response from server
byte[] response = new byte[client.ReceiveBufferSize];
byte[] bufferSize = new byte[1024];
int bytesRead;
while ((bytesRead = serverStream.Read(bufferSize, 0, bufferSize.Length)) > 0 && client.ReceiveBufferSize > 0)
{
Debug.WriteLine("Bytes read: " + bytesRead);
fs.Write(response, 0, bytesRead);
}
fs.Close();

With UDP you can transmit an effectively empty packet, but TCP won't allow you to do that. At the application layer the TCP protocol is a stream of bytes, with all of the packet-level stuff abstracted away. Sending zero bytes will not result in anything happening at the stream level on the client side.
Signalling the end of a file transfer can be as simple as having the server close the connection after sending the last block of data. The client will receive the final data packet then note that the socket has been closed, which indicates that the data has been completely delivered. The flaw in this method is that the TCP connection can be closed for other reasons, leaving a client in a state where it believes that it has all the data even though the connection was dropped for another reason.
So even if you are going to use the 'close on complete' method to signal end of transfer, you need to have a mechanism that allows the client to identify that the file is actually complete.
The most common form of this is to send a header block at the start of the transfer that tells you something about the data being transferred. This might be as simple as a 4-byte length value, or it could be a variable-length descriptor structure that includes various metadata about the file such as its length, name, create/modify times and a checksum or hash that you can use to verify the received content. The client reads the header first, then processes the rest of the data in the stream as content.
Let's take the simplest case, sending a 4-byte length indicator at the start of the stream.
Server Code:
public void SendStream(Socket client, Stream data)
{
// Send length of stream as first 4 bytes
byte[] lenBytes = BitConverter.GetBytes((int)data.Length);
client.Send(lenBytes);
// Send stream data
byte[] buffer = new byte[1024];
int rc;
data.Position = 0;
while ((rc = data.Read(buffer, 0, 1024)) > 0)
client.Send(buffer, rc, SocketFlags.None);
}
Client Code:
public bool ReceiveStream(Socket server, Stream outdata)
{
// Get length of data in stream from first 4 bytes
byte[] lenBytes = new byte[4];
if (server.Receive(lenBytes) < 4)
return false;
long len = (long)BitConverter.ToInt32(lenBytes, 0);
// Receive remainder of stream data
byte[] buffer = new byte[1024];
int rc;
while ((rc = server.Receive(buffer)) > 0)
outdata.Write(buffer, 0, rc);
// Check that we received the expected amount of data
return len == outdata.Position;
}
Not much in the way of error checking and so on, and blocking code in all directions, but you get the idea.

There is no such thing as sending "zero bytes" in a stream. As soon as the stream sees you're trying to send zero bytes it can just return immediately and will have done exactly what you asked.
Since you're using TCP, it is up to you to use an agreed-upon protocol between the client and server. For example:
The server could close the connection after sending all its data. The client would see this as a "Read" that completes with zero bytes returned.
The server could send a header of a fixed size (maybe 4 bytes) that includes the length of the upcoming data. The client could then read those 4 bytes and would then know how many more bytes to wait for.
Finally, you might need a "netStream.Flush()" in your server code above (if you intended to keep the connection open).

Read bytes from NetworkStream (Hangs)

I'm trying to learn the basics of networking and I've built an echo server from this tutorial. I checked the server with telnet and it works perfect.
Now when I'm using some of the many client samples on the Internet:
// Create a TcpClient.
// Note, for this client to work you need to have a TcpServer
// connected to the same address as specified by the server, port
// combination.
TcpClient client = new TcpClient(server, port);
// Translate the passed message into ASCII and store it as a Byte array.
Byte[] data = System.Text.Encoding.ASCII.GetBytes(message);
// Get a client stream for reading and writing.
NetworkStream stream = client.GetStream();
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
Console.WriteLine("Sent: {0}", message);
// Receive the TcpServer.response.
// Buffer to store the response bytes.
data = new Byte[256];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
Console.WriteLine("Received: {0}", responseData);
// Close everything.
stream.Close();
client.Close();
It doesn't work very well. If I will comment the stream.Read line, everything works perfect (expect I can't read). I was also trying to accomplish that in a similar way using asynchronous callback method for the read. and then it only works after I terminate the program (the server handles the request)
I suspect that the way I'm reading from the stream cause this block, but I'm too clueless to understand what I'm doing wrong.

The implementation will block until at least one byte of data can be
read, in the event that no data is available.
From MSDN
Your server propably isn't sending you any data.
Edit:
I tested your client and it works perfectly fine. Try it yourself and set the following parameters:
string server = "google.com";
int port = 80;
string message = "GET /\n";
It's definitely your server which has the problem.

Send server multiple messages? C#

I have a quick and dirty question. So as it stands, i have two clients and a server running. I can communicate messages from the clients to the server without any problem. my problem appears when i want to read two messages from the client - rather than just one message.
The error which i receive is: IOException was unhandled. Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
Here is my code on the server side:
private static void HandleClientComm(object client)
{
/** creating a list which contains DatabaseFile objects **/
List theDatabase = new List();
TcpClient tcpClient = (TcpClient)client;
NetworkStream clientStream = tcpClient.GetStream();
byte[] message = new byte[4096];
int bytesRead;
do
{
bytesRead = 0;
try
{
// Blocks until a client sends a message
bytesRead = clientStream.Read(message, 0, 4096);
}
catch (Exception)
{
// A socket error has occured
break;
}
if (bytesRead == 0)
{
// The client has disconnected from the server
break;
}
// Message has successfully been received
ASCIIEncoding encoder = new ASCIIEncoding();
Console.WriteLine("To: " + tcpClient.Client.LocalEndPoint);
Console.WriteLine("From: " + tcpClient.Client.RemoteEndPoint);
Console.WriteLine(encoder.GetString(message, 0, bytesRead));
if (encoder.GetString(message, 0, bytesRead) == "OptionOneInsert")
{
byte[] message2 = new byte[4096];
int bytesRead2 = 0;
**bytesRead2 = clientStream.Read(message, 0, 4096);** //ERROR occurs here!
Console.WriteLine("Attempting to go inside insert)");
Menu.Insert(theDatabase, bytesRead2);
}
Here is my client code:
ASCIIEncoding encoder = new ASCIIEncoding();
byte[] buffer = encoder.GetBytes("OptionOneInsert");
Console.ReadLine();
clientStream.Write(buffer, 0, buffer.Length);
clientStream.Flush();
NetworkStream clientStream2 = client.GetStream();
String text = System.IO.File.ReadAllText("FirstNames.txt");
clientStream2.Write(buffer, 0, buffer.Length);
clientStream2.Flush();
ASCIIEncoding encoder2 = new ASCIIEncoding();
byte[] buffer2 = encoder2.GetBytes(text);
Console.WriteLine("buffer is filled with content");
Console.ReadLine();
When the client sends the message "optionOne" it is received by the server just fine. It's only when i attempt to send the string called "text" that the issues appears!
Any help would be greatly appreciated - I'm not all that familiar with Sockets, hence i've been struggling with trying to understand this for sometime now

You've got a big problem here - there's nothing to specify the end of one message and the start of another. It's quite possible that the server will receive two messages in one go, or half a message and then the other half.
The simplest way of avoiding that is to prefix each message with the number of bytes in it, e.g. as a fixed four-byte format. So to send a message you would:
Encoding it from a string to bytes (ideally using UTF-8 instead of ASCII unless you're sure you'll never need any non-ASCII text)
Write out the length of the byte array as a four-byte value
Write out the content
On the server:
Read four bytes (looping if necessary - there's no guarantee you'd even read those four bytes together, although you almost certainly will)
Convert the four bytes into an integer
Allocate a byte array of that size
Loop round, reading from "the current position" to the end of the buffer until you've filled the buffer
Convert the buffer into a string
Another alternative is simply to use BinaryReader and BinaryWriter - the ReadString and WriteString use length-prefixing, admittedly in a slightly different form.
Another alternative you could use is to have a delimiter (e.g. carriage-return) but that means you'll need to add escaping in if you ever need to include the delimiter in the text to transmit.