We Keep Coding

NetworkStream.read taking 2 seconds to read data from stream (TCP C#)

I have a TCP request response model in C# where I am communicating with a server. Once the server has written data to the stream, I am reading that data. But stream.read is taking 2 seconds to read the data. I need to send an explicit acknowledgement to the server, within 2 seconds but am unable to do so because of the time taken to read the data.
Below is my code to read data:
byte[] resp = new byte[100000];
var memoryStream = new MemoryStream();
int bytes;
String timeStamp = GetTimestamp(DateTime.Now);
Console.WriteLine("Before reading data: ");
Console.WriteLine(timeStamp);
do
{
bytes = stream.Read(resp, 0, resp.Length);
memoryStream.Write(resp, 0, bytes);
}
while (bytes > 0);
timeStamp = GetTimestamp(DateTime.Now);
Console.WriteLine("After reading data: ");
Console.WriteLine(timeStamp);
GenerateAcknowledgemnt(stream);
timeStamp = GetTimestamp(DateTime.Now);
Console.WriteLine("After sending ack: ");
Console.WriteLine(timeStamp);
Below are the timestamps read, in the format yyyyMMddHHmmssff:
Before reading data:
2022050615490817
After reading data:
2022050615491019
After sending ack:
2022050615491020
I have highlighted the seconds bold.
How do I reduce the time that stream.read is taking to read? I have tried to wrap the network stream in a BufferedStream as well, but it didn't help.

At the moment, you are performing a read loop that keeps going until Read returns a non-positive number; in TCP, this means you are waiting until the other end hangs up (or at least hangs up their outbound socket) until you get out of that loop. I suspect what is happening is that the other end is giving up on you, closing their connection, and only then do you get out of the loop.
Basically: you can't loop like that; instead, what you need to do is to carefully read until either EOF (bytes <= 0) or until you have at least one complete frame that you can respond to, and in the latter case: respond then. This usually means a loop more like (pseudo-code):
while (TryReadSomeMoreData()) // performs a read into the buffer, positive result
{
// note: may have more than one frame per successful 'read'
while (TryParseOneFrame(out frame))
{
ProcessFrame(frame); // includes sending responses
// (and discard anything that you've now processed from the back-buffer)
}
}
(parsing a frame here means: following whatever rules apply about isolating a single message from the stream - this may mean looking for a sentinel value such as CR/LF/NUL, or may mean checking if you have enough bytes to read a header that includes a length, and then checking that you have however-many bytes the header indicates as the payload)
This is a little awkward if you're using MemoryStream as the backlog, as the discard step is not convenient; the "pipelines" API is more specifically designed for this, but: either way can work.
Secondly: you may prefer async IO, although sync IO is probably fine for a simple client application with only one connection (but not for servers, which may have many many connections).

Block incoming data from C# NetworkStream after timeout has passed

I have a strange question that involves NetworkStreams in C# .Net Core 2.2.101
My setup is as follows:
I have a list of meters
Each meter has a list of registers (a register saves a value, for example: the voltage or current)
Meters are connected to a GSM modem via RS485 (irrelevant for the question)
I send commands to the modem to read specific registers for specific meters
For each register of each meter, I send a command using stream.Write(bytesToSend, 0, bytesToSend.Length); to ask the meter to send me the data that is saved in a specific register in the meter. Directly after sending, I read the response using stream.Read(buffer, 0, buffer.Length);. I also set a read timeout of 5 seconds, which will block and wait for 5 seconds before moving on to the next register if no response has been received before the timeout.
The problem:
What is happening is that when I ask the meter for data, sometimes it will take too long and the timeout will be reached, after which I will move on to ask the next register for data, but then sometimes the first register will reply with data after I have moved on to the next register (meaning that the NetworkStream now has the data from the previous register). Since I have already moved on in my for-loop, my program thinks that the data I am reading from the stream is for the current register, when in fact it is for the previous register from the previous iteration. This messes up the data that goes into the database, because I am saving the wrong value for the wrong register.
My question is: Is there a clever way to ignore any incoming data from a previous iteration in my for-loop? Unfortunately there is no information in the data received that could be used to identify for which register the data is for.
Here is a snippet of what my write and read requests look like:
stream.ReadTimeout = 5000;
stream.WriteTimeout = 2000;
foreach (Meter meter in metersToRead)
{
foreach (Register register in meter.Registers)
{
// Write the request to the meter
stream.Write(bytesToSend, 0, bytesToSend.Length);
// Read response from meter
requestedReadingDataCount = stream.Read(buffer, 0, buffer.Length);
// Extract the value from the response buffer and save the value to the database
...
}
}
I want to try and clone the stream and use the cloned stream for communication regarding the current register iteration, so that when a response comes in after I have closed the cloned stream and moved on to the next register, the response will fail since the stream has been closed. However, I am not sure if you can clone a C# NetworkStream? Anybody know?
My last resort will be to make a call to the database after I read the data for each register, to check if the data I received is reasonable for that register, but I fear that this might slow the program down with all the database calls and I would have to build up some rules that will determine whether a value is reasonable for the current register.
Any ideas would be greatly appreciated. If you have any questions, please let me know and I will try my best to explain it further.
Edit
Here is an updated code snippet, as well as an image that will better explain the issue that I am having.
private async Task ReadMeterRegisters(List<MeterWithRegisters> metersWithRegisters, NetworkStream stream)
{
stream.ReadTimeout = 5000; /* Read timeout set to 5 seconds */
stream.WriteTimeout = 2000; /* Write timeout set to 2 seconds */
foreach (Meter meter in metersToRead)
{
foreach (Register register in meter.Registers)
{
// Instantiate a new buffer to hold the response
byte[] readingResponseDataBuffer = new byte[32];
// Variable to hold number of bytes received
int numBytesReceived = 0;
try
{
// Write the request to the meter
stream.Write(bytesToSend, 0, bytesToSend.Length);
// Read response from meter
numBytesReceived = stream.Read(buffer, 0, buffer.Length);
}
catch (IOException) /* catch read/write timeouts */
{
// No response from meter, move on to next register of current meter
continue;
}
// Extract the value from the response buffer and save the value to the database
...
}
}
}

It sounds like the issue here is that in a timeout scenario, the read operation is still completing at some point in the future but writing to the old buffer. If that is the case, perhaps the simplest option is to not reuse the read buffer in the event of a timeout (meaning: assign a new byte[] to buffer), and consider that network-stream burned (since you now can't know what the internal state is).
An alternative approach would be to not read until you know there is data; you can't do that from NetworkStream, but on Socket you can check .Available to see whether there is data to be read; that way, you won't be performing ambiguous reads. You can also perform a zero-length read on a socket (at least on most OS-es); if you pass a zero-length buffer, it will block until either the timeout or until data becomes available, but without consuming any data (the idea being that you follow a zero-length read with a non-zero-length read if you find that data has become available).
In the more general case: you might find you get better throughput here if you switch to asynchronous IO rather than synchronous IO; you could even use the array-pool for the buffers. For dealing with large volumes of connections, async IO is almost always the way to go.

Send / receive file over TCP [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
TcpClient send data and receive data over network
Loop until TcpClient response fully read
I am trying to send a file from a server to a client over TCP.
Server-side code, sending file:
NetworkStream netStream = client.GetStream();
FileStream fs = new FileStream("usb.exe",FileMode.Open, FileAccess.Read);
byte[] data = new byte[fs.Length];
fs.Read(data,0, data.Length);
fs.Flush();
fs.Close();
netStream.Write(data, 0, data.Length);
netStream.Flush();
Client-side code, receive file:
FileStream str = new FileStream("usb.exe", FileMode.Create, FileAccess.Write);
byte[] data = new byte[1024];
while ((dataCitit = netStream.Read(data,0, data.Length)) > 0)
{
Thread.Sleep(25);
Application.DoEvents();
str.Write(data, 0, dataCitit);
totalbytes += dataCitit;
}
str.Close();
Can someone point where I am getting it wrong ?
The file has 1036 kb, and it sends only 1032 kb and then gets stuck it won't get out the while loop on the client-side.
Also if I close the server and open it really quick it sends the last bytes and the files sends completely. (this file opens perfectly)
I think it`s a problem on the server side not sending all the bytes but why and where...

Well this is a problem in your server-side code to start with:
fs.Read(data,0, data.Length);
You're ignoring the value returned by Read. Never do that. With FileStream you're possibly okay, but I personally wouldn't trust it anyway. If you're using .NET 4, you don't need to do this anyway - just use Stream.CopyTo.
On the client-side code, your biggest initial problem is that you're doing all of this on the UI thread. That's a terrible idea - the UI will freeze if there's a network glitch, as the Read call is blocking.
Again, just use Stream.CopyTo, but do it in a background thread.
Additionally, in all of these cases, use a using statement for the streams, so that you close them cleanly whatever happens.
That's all just general hygiene. Now, as for why you're hanging...
... you're not closing the network stream on the server side. Therefore you never reach the end of the stream on the client side. If you only need to use the connection for a single file, then the answer is simple: just close the connection on the server side.
If, however, you need to use the same connection for multiple files, then you need more protocol - you need to some way of indicating the end of the data. There are three common ways of doing that:
Write the length of the data before the data itself, then on the reading side, first read the length, then read that many bytes, failing if the stream finishes before you've done so. This requires that you know how much data you're going to write before you start writing.
Use an "end of data" marker which you can detect on the reading side; this is a pain in general, as it requires escaping the marker if it appears in the text itself.
A variation on the first approach, where you write a length-prefixed chunk at a time, then a zero-length chunk to indicate "end of data". This is pretty flexible, but obviously a bit more work than the first approach if the first approach actually works for you.

How to send files over tcp with TcpListener/Client? SocketException problem

I'm developing a simple application to send files over TCP using the TCPListener and TCPClient classes. Here's the code that sends the file.
Stop is a volatile boolean which helps stopping the process at any time and WRITE_BUFFER_SIZE might be changed in runtime (another volatile)
while (remaining > 0 && !stop)
{
DateTime current = DateTime.Now;
int bufferSize = WRITTE_BUFFER_SIZE;
buffer = new byte[bufferSize];
int readed = fileStream.Read(buffer, 0, bufferSize);
stream.Write(buffer, 0, readed);
stream.Flush();
remaining -= readed;
// Wait in order to guarantee send speed
TimeSpan difference = DateTime.Now.Subtract(current);
double seconds = (bufferSize / Speed);
int wait = (int)Math.Floor(seconds * 1000);
wait -= difference.Milliseconds;
if (wait > 10)
Thread.Sleep(wait);
}
stream.Close();
and this is the code that handles the receiver side:
do
{
readed = stream.Read(buffer, 0, READ_BUFFER_SIZE);
// write to .part file and flush to disk
outputStream.Write(buffer, 0, readed);
outputStream.Flush();
offset += readed;
} while (!stop && readed > 0);
Now, when the speed is low (about 5KBps) everything works ok but, as I increase the speed the receiver size becomes more prone to raise a SocketException when reading from the stream. I'm guessing it has to do with the remote socket being closed before all data can be read, but What's the correct way to do this? When should I close the sending client?
I haven't found any good examples of file transmission on google, and the ones that I've found have a similar implementation of what I'm doing so I guess I'm missing something.
Edit: I get this error "Unable to read data from the transport connection". This is an IOException whose inner exception is a SocketException.
I've added this in the sender function, still I get the same error, the code never reaches the stream.close() and of course the tcpclient never really get closed... so I'm completely lost now.
buffer = new byte[1];
client.Client.Receive(buffer);
stream.Close();

Typically you want to set the LINGER option on the socket. Under C++ this would be SO_LINGER, but under windows this doesn't actually work as expected. You really want to do this:
Finish sending data.
Call shutdown() with the how parameter set to 1.
Loop on recv() until it returns 0.
Call closesocket().
Taken from: http://tangentsoft.net/wskfaq/newbie.html#howclose
C# sharp may have corrected this in its libraries, but I doubt it since they are built on top of the winsock API.
Edit:
Looking at your code in more detail. I see that you are sending no header across at all, so on the receiving side you have no idea of how many bytes you are actually supposed to read. Knowing the number of bytes to read of the socket makes this a much easier problem to debug. Keep in mind that shutting down the socket can still snip of the last bit of data if you don't close it properly.
Additionally having your buffer size be volatile is not thread safe and really doesn't buy you anything. Using stop as a volatile is safe, but don't expect it to be instant. In other words the loop could run several more times before it gets the updated value of stop. This is especially true on multiprocessor machines.
Edit_02:
For the TCPClientClass you want to do the following (as far as I can tell without having access to a C# at the moment).
// write all the bytes
// Then do the following
client.client.Shutdown(Shutdown.Send) // This assumes you have access to this protected member
while (stream.read(buffer, 0, READ_BUFFER_SIZE) != 0);
client.close()

How to Call NetworkStream.Read() Without Blocking?

I'd like to empty read buffer of the socket so I wrote follow code...
byte[] tempBuffer = new byte[1024];
int readCount = 0;
while ((readCount = tcpSocket.GetStream().Read(tempBuffer, 0, tempBuffer.Length)) != 0)
{
// do with tempBuffer
}
But Read() method is blocked so I added tcpSocket.ReceiveTimeout = 1;. And it works just like before.
As I know, this is usually used in C++. How can I solve this problem?

You can use the DataAvailable property to see if there is anything to be read before making a call into the Read method.

Use the NetworkStream.Read() function directly, instead of using GetStream():
If no data is available for reading,
the Read method returns 0. The Read
operation reads as much data as is
available, up to the number of bytes
specified by the size parameter. If
the remote host shuts down the
connection, and all available data has
been received, the Read method
completes immediately and return zero
bytes. NoteNote:

Why do you want to empty the read buffer?
If you don't want the contents of the socket close it.
If you don't want the current contents, but will want later data, how do you know when later starts. If the data is an non-encapsulated stream...
Sounds like your solving the problem in the wrong fashion.