We Keep Coding

Block incoming data from C# NetworkStream after timeout has passed

I have a strange question that involves NetworkStreams in C# .Net Core 2.2.101
My setup is as follows:
I have a list of meters
Each meter has a list of registers (a register saves a value, for example: the voltage or current)
Meters are connected to a GSM modem via RS485 (irrelevant for the question)
I send commands to the modem to read specific registers for specific meters
For each register of each meter, I send a command using stream.Write(bytesToSend, 0, bytesToSend.Length); to ask the meter to send me the data that is saved in a specific register in the meter. Directly after sending, I read the response using stream.Read(buffer, 0, buffer.Length);. I also set a read timeout of 5 seconds, which will block and wait for 5 seconds before moving on to the next register if no response has been received before the timeout.
The problem:
What is happening is that when I ask the meter for data, sometimes it will take too long and the timeout will be reached, after which I will move on to ask the next register for data, but then sometimes the first register will reply with data after I have moved on to the next register (meaning that the NetworkStream now has the data from the previous register). Since I have already moved on in my for-loop, my program thinks that the data I am reading from the stream is for the current register, when in fact it is for the previous register from the previous iteration. This messes up the data that goes into the database, because I am saving the wrong value for the wrong register.
My question is: Is there a clever way to ignore any incoming data from a previous iteration in my for-loop? Unfortunately there is no information in the data received that could be used to identify for which register the data is for.
Here is a snippet of what my write and read requests look like:
stream.ReadTimeout = 5000;
stream.WriteTimeout = 2000;
foreach (Meter meter in metersToRead)
{
foreach (Register register in meter.Registers)
{
// Write the request to the meter
stream.Write(bytesToSend, 0, bytesToSend.Length);
// Read response from meter
requestedReadingDataCount = stream.Read(buffer, 0, buffer.Length);
// Extract the value from the response buffer and save the value to the database
...
}
}
I want to try and clone the stream and use the cloned stream for communication regarding the current register iteration, so that when a response comes in after I have closed the cloned stream and moved on to the next register, the response will fail since the stream has been closed. However, I am not sure if you can clone a C# NetworkStream? Anybody know?
My last resort will be to make a call to the database after I read the data for each register, to check if the data I received is reasonable for that register, but I fear that this might slow the program down with all the database calls and I would have to build up some rules that will determine whether a value is reasonable for the current register.
Any ideas would be greatly appreciated. If you have any questions, please let me know and I will try my best to explain it further.
Edit
Here is an updated code snippet, as well as an image that will better explain the issue that I am having.
private async Task ReadMeterRegisters(List<MeterWithRegisters> metersWithRegisters, NetworkStream stream)
{
stream.ReadTimeout = 5000; /* Read timeout set to 5 seconds */
stream.WriteTimeout = 2000; /* Write timeout set to 2 seconds */
foreach (Meter meter in metersToRead)
{
foreach (Register register in meter.Registers)
{
// Instantiate a new buffer to hold the response
byte[] readingResponseDataBuffer = new byte[32];
// Variable to hold number of bytes received
int numBytesReceived = 0;
try
{
// Write the request to the meter
stream.Write(bytesToSend, 0, bytesToSend.Length);
// Read response from meter
numBytesReceived = stream.Read(buffer, 0, buffer.Length);
}
catch (IOException) /* catch read/write timeouts */
{
// No response from meter, move on to next register of current meter
continue;
}
// Extract the value from the response buffer and save the value to the database
...
}
}
}

It sounds like the issue here is that in a timeout scenario, the read operation is still completing at some point in the future but writing to the old buffer. If that is the case, perhaps the simplest option is to not reuse the read buffer in the event of a timeout (meaning: assign a new byte[] to buffer), and consider that network-stream burned (since you now can't know what the internal state is).
An alternative approach would be to not read until you know there is data; you can't do that from NetworkStream, but on Socket you can check .Available to see whether there is data to be read; that way, you won't be performing ambiguous reads. You can also perform a zero-length read on a socket (at least on most OS-es); if you pass a zero-length buffer, it will block until either the timeout or until data becomes available, but without consuming any data (the idea being that you follow a zero-length read with a non-zero-length read if you find that data has become available).
In the more general case: you might find you get better throughput here if you switch to asynchronous IO rather than synchronous IO; you could even use the array-pool for the buffers. For dealing with large volumes of connections, async IO is almost always the way to go.

C# streamreader readtoend freeze up

I have the following code to read the time from time.nist.gov:
TcpClient client = new TcpClient();
var result = client.BeginConnect("129.6.15.28", 13, null, null);
var success = result.AsyncWaitHandle.WaitOne(TimeSpan.FromSeconds(1));
if (!success || !client.Connected)
// Timeout
else{
streamReader = new StreamReader(client.GetStream());
var response = streamReader.ReadToEnd();
//code
}
The problem is that sometimes ReadToEnd get frozen indefinitely, I assume because it does not reach and end.
Is it possible to set a timeout for this operation?
Or maybe a better option to read the server response?
Thanks.

According to MSDN:
ReadToEnd assumes that the stream knows when it has reached an end. For interactive protocols in which the server sends data only when you ask for it and does not close the connection, ReadToEnd might block indefinitely because it does not reach an end, and should be avoided.
Do you know how much you're reading? If so, you can use Read().
EDIT:
I tested this code (well, something very similar) and found that ReadToEnd works fine, unless port 13 is blocked, in which case it also runs without returning for me as well. So, check your firewall before doing this.
Better is Hans's suggestion of using an existing, well-tested NTP lib.

Send / receive file over TCP [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
TcpClient send data and receive data over network
Loop until TcpClient response fully read
I am trying to send a file from a server to a client over TCP.
Server-side code, sending file:
NetworkStream netStream = client.GetStream();
FileStream fs = new FileStream("usb.exe",FileMode.Open, FileAccess.Read);
byte[] data = new byte[fs.Length];
fs.Read(data,0, data.Length);
fs.Flush();
fs.Close();
netStream.Write(data, 0, data.Length);
netStream.Flush();
Client-side code, receive file:
FileStream str = new FileStream("usb.exe", FileMode.Create, FileAccess.Write);
byte[] data = new byte[1024];
while ((dataCitit = netStream.Read(data,0, data.Length)) > 0)
{
Thread.Sleep(25);
Application.DoEvents();
str.Write(data, 0, dataCitit);
totalbytes += dataCitit;
}
str.Close();
Can someone point where I am getting it wrong ?
The file has 1036 kb, and it sends only 1032 kb and then gets stuck it won't get out the while loop on the client-side.
Also if I close the server and open it really quick it sends the last bytes and the files sends completely. (this file opens perfectly)
I think it`s a problem on the server side not sending all the bytes but why and where...

Well this is a problem in your server-side code to start with:
fs.Read(data,0, data.Length);
You're ignoring the value returned by Read. Never do that. With FileStream you're possibly okay, but I personally wouldn't trust it anyway. If you're using .NET 4, you don't need to do this anyway - just use Stream.CopyTo.
On the client-side code, your biggest initial problem is that you're doing all of this on the UI thread. That's a terrible idea - the UI will freeze if there's a network glitch, as the Read call is blocking.
Again, just use Stream.CopyTo, but do it in a background thread.
Additionally, in all of these cases, use a using statement for the streams, so that you close them cleanly whatever happens.
That's all just general hygiene. Now, as for why you're hanging...
... you're not closing the network stream on the server side. Therefore you never reach the end of the stream on the client side. If you only need to use the connection for a single file, then the answer is simple: just close the connection on the server side.
If, however, you need to use the same connection for multiple files, then you need more protocol - you need to some way of indicating the end of the data. There are three common ways of doing that:
Write the length of the data before the data itself, then on the reading side, first read the length, then read that many bytes, failing if the stream finishes before you've done so. This requires that you know how much data you're going to write before you start writing.
Use an "end of data" marker which you can detect on the reading side; this is a pain in general, as it requires escaping the marker if it appears in the text itself.
A variation on the first approach, where you write a length-prefixed chunk at a time, then a zero-length chunk to indicate "end of data". This is pretty flexible, but obviously a bit more work than the first approach if the first approach actually works for you.

C# Begin Send within a foreach loop issue

I have a group of "Packets" which are custom classed that are coverted to byte[] and then sent to the client. When a client joins, they are updated with the previous "Catch Up Packets" that were sent previous to the user joining. Think of it as a chat room where you are updated with the previous conversations.
My issue is on the client end, we do not receive all the information; Sometimes not at all..
Below is pseudo c# code for what I see
code looks like this.
lock(CatchUpQueue.SyncRoot)
{
foreach(Packet packet in CatchUpQueue)
{
// If I put Console.WriteLine("I am Sending Packets"); It will work fine up to (2) client sockets else if fails again.
clientSocket.BeginSend(data, 0, data.length, SocketFlags.None, new AsyncCallback(EndSend), data);
}
}
Is this some sort of throttle issue or an issue with sending to many times: ie: if there are 4 packets in the queue then it calls begin send 4 times.
I have searched for a topic similiar and I cannot find one. Thank you for your help.
Edit: I would also like to point out that the sending between clients continues normally for any sends after the client connects. But for some reason the packets within this for loop are not all sent.

I would suspect that you are flooding the TCP port with packets, and probably overflowing its send buffer, at which point it will probably return errors rather than sending the data.
The idea of Async I/O is not to allow you to send an infinite amount of data packets simultaneously, but to allow your foreground thread to continue processing while a linear sequence of one or more I/O operations occurs in the background.
As the TCP stream is a serial stream, try respecting that and send each packet in turn. That is, after BeginSend, use the Async callback to detect when the Send has completed before you send again. You are effectively doing this by adding a Sleep, but this is not a very good solution (you will either be sending packets more slowly than possible, or you may not sleep for long enough and packets will be lost again)
Or, if you don't need the I/O to run in the background, use your simple foreach loop, but use a synchronous rather than Async send.

Okay,
Apparently a fix, so far still has me confused, is to Thread.Sleep for the number of ms for each packet I am sending.
So...
for(int i = 0; i < PacketQueue.Count; i++)
{
Packet packet = PacketQueue[i];
clientSocket.BeginSend(data, 0, data.length, SocketFlags.None, new AsyncCallback(EndSend), data);
Thread.Sleep(PacketQueue.Count);
}
I assume that for some reason the loop stops some of the calls from happening... Well I will continue to work with this and try to find the real answer.

How to determine if an HTTP response is complete

I am working on building a simple proxy which will log certain requests which are passed through it. The proxy does not need to interfere with the traffic being passed through it (at this point in the project) and so I am trying to do as little parsing of the raw request/response as possible durring the process (the request and response are pushed off to a queue to be logged outside of the proxy).
My sample works fine, except for a cannot reliably tell when the "response" is complete so I have connections left open for longer than needed. The relevant code is below:
var request = getRequest(url);
byte[] buffer;
int bytesRead = 1;
var dataSent = false;
var timeoutTicks = DateTime.Now.AddMinutes(1).Ticks;
Console.WriteLine(" Sending data to address: {0}", url);
Console.WriteLine(" Waiting for response from host...");
using (var outboundStream = request.GetStream()) {
while (request.Connected && (DateTime.Now.Ticks < timeoutTicks)) {
while (outboundStream.DataAvailable) {
dataSent = true;
buffer = new byte[OUTPUT_BUFFER_SIZE];
bytesRead = outboundStream.Read(buffer, 0, OUTPUT_BUFFER_SIZE);
if (bytesRead > 0) { _clientSocket.Send(buffer, bytesRead, SocketFlags.None); }
Console.WriteLine(" pushed {0} bytes to requesting host...", _backBuffer.Length);
}
if (request.Connected) { Thread.Sleep(0); }
}
}
Console.WriteLine(" Finished with response from host...");
Console.WriteLine(" Disconnecting socket");
_clientSocket.Shutdown(SocketShutdown.Both);
My question is whether there is an easy way to tell that the response is complete without parsing headers. Given that this response could be anything (encoded, encrypted, gzip'ed etc), I dont want to have to decode the actual response to get the length and determine if I can disconnect my socket.

As David pointed out, connections should remain open for a period of time. You should not close connections unless the client side does that (or if the keep alive interval expires).
Changing to HTTP/1.0 will not work since you are a server and it's the client that will specify HTTP/1.1 in the request. Sure, you can send a error message with HTTP/1.0 as version and hope that the client changes to 1.0, but it seems inefficient.
HTTP messages looks like this:
REQUEST LINE
HEADERS
(empty line)
BODY
The only way to know when a response is done is to search for the Content-Length header. Simply search for "Content-Length:" in the request buffer and extract everything to the linefeed. (But trim the found value before converting to int).
The other alternative is to use the parser in my webserver to get all headers. It should be quite easy to use just the parser and nothing more from the library.
Update: There is a better parser here: HttpParser.cs

If you make a HTTP/1.0 request instead of 1.1, the server should close the connection as soon as it's through since it doesn't need to keep the connection open for another request.
Other than that, you really need to parse the content length header in the response to get the best value.

Using blocking IO and multiple threads might be your answer. Specifically
using(var response = request.GetResponse())
using(var stream = response.GetResponseStream())
using(var reader = new StreamReader(stream)
data = reader.ReadToEnd()
This is for textual data, however binary handling is similar.