We Keep Coding

How do I properly handle all edge cases when reading from a PipeReader in ASP.NET Core?

I'm reading the body of a request in ASP.NET Core 3.1, using Request.BodyReader (which is a PipeReader instance).
In tests, the following seems to work, but frankly the documentation for PipeReader has a couple of warnings in it that I don't understand, so I'm not sure the following is correct.
public static async Task ParseRequest(PipeReader bodyReader, CancellationToken requestAborted) {
while (true) {
var result = await bodyReader.ReadAsync(requestAborted);
ReadOnlySequence<byte> buffer = result.Buffer;
if (result.IsCompleted) {
try
{
ProcessRequest(buffer.IsSingleSegment ? buffer.FirstSpan : buffer.ToArray().AsSpan());
}
catch (Exception e) {
Logger.Log(e);
}
}
if (result.IsCompleted || result.IsCanceled || buffer.Length > 1000)
{
//If result.IsCompleted, does this matter?
//The code here doesn't fully advance if IsCompleted: https://learn.microsoft.com/en-us/aspnet/core/fundamentals/middleware/request-response?view=aspnetcore-5.0
bodyReader.AdvanceTo(buffer.End);
break;
}
else
{
bodyReader.AdvanceTo(buffer.Start, buffer.End);
}
}
//await bodyReader.CompleteAsync(); //<-- is this needed? or not my responsibility?
}
Design goals: When the whole body is available, parse the whole thing via ProcessRequest(). If the body is longer than 1000 bytes, ignore the request.
Questions:
Do I need to call CompleteAsync() on the PipeReader? I looked at the source code for FormPipeReader, and it never calls Complete on the reader, so I assume this is not necessary (i.e. is done by the framework).
The Pipelines documentation warns: "Unconditionally calling PipeReader.AdvanceTo with buffer.End in the examined position may result in hangs when parsing a single message. The next call to PipeReader.AdvanceTo won't return until:
1) There's more data written to the pipe.
2) And the new data wasn't previously examined."
I don't understand what scenario they're describing here. What are the conditions where you should not call PipeReader.AdvanceTo(consumed, buffer.End) when you need more data? In the correct use examples in the same article, they have that exact call in a finally {} block, i.e. unconditionally calling it.
Should I call AdvanceTo(buffer.End) when a body has been read, and I've parsed it completely? The example code here does not do so when result.IsCompleted is true, yet the docs also say that not advancing to the end can result in a memory leak. Is it that some other code in ASP.NET ends up taking care of fully consuming the pipe?

C# Named Pipes, how to detect a client disconnecting

My current named pipe implementation reads like this:
while (true)
{
byte[] data = new byte[256];
int amount = pipe.Read(data, 0, data.Length);
if (amount <= 0)
{
// i was expecting it to go here when a client disconnects but it doesnt
break;
}
// do relevant stuff with the data
}
how can I correctly detect when a client disconnects?

Set a read timeout and poll the NamedPipeClientStream.IsConnected flag when a timeout occurs.
A Read Timeout will cause reads that are idle for the timeout duration to throw InvalidOperationException
If you are not reading, and want to detect disconnections, call this method on a worker thread for the lifetime of your pipe connection.
while(pipe.IsConnected && !isPipeStopped) //use a flag so that you can manually stop this thread
{
System.Threading.Thread.Current.Sleep(500);
}
if(!pipe.IsConnected)
{
//pipe disconnected
NotifyOfDisconnect();
}

One easy way to tell if your pipe has been broken (remotely) is to always use asynchronous reads instead of sync ones, and to always have at least one read submitted asynchronously. That is, for every successful read you get, post another async read, whether you intend to read another or not. If you close the pipe, or the remote end closes it, you'll see the async read complete, but with a null read size. You can use this to detect a pipe disconnection. Unfortunately, the pipe will still show IsConnected, and you still need to manually close it, I think, but it does allow you to detect when something went wonky.

Use WaitForPipeDrain() method after Writing to the Pipe (using WriteByte() or Write()) and catch the exception which is "Pipe is Broken".
You may want to put that in a while loop and keep writing to the pipe.

in the case of Synchronous call you track the -1 return by ReadByte of Stream abstract class, which is inherited by NamedPipeServerStream:
var _pipeServer = new NamedPipeServerStream(PipeConst._PIPE_NAME, PipeDirection.InOut);
int firstByte = _pipeServer.ReadByte();
const int END_OF_STREAM = -1;
if (firstByte == END_OF_STREAM)
{
return null;
}
The docs states indeed:
//
// Summary:
// Reads a byte from a pipe.
//
// Returns:
// The byte, cast to System.Int32, or -1 indicates the end of the stream (the pipe
// has been closed).
public override int ReadByte();
Only after a first failed read your IsConnected property will be correctly set to false:
_pipeServer.IsConnected
You might observe that even on the
official illustration of Microsoft (and more precisely in the StreamString class)
this check is not done:
Do not forget to Vote for this answer and visit my Youtube channel. More info on my profile.
Regards !

What is the best approach for serial data reception and processing using c#?

I am pretty new to coding with some experience in ASM and C for PIC. I am still learning high level programming with C#.
Question
I have a Serial port data reception and processing program in C#. To avoid losing data and knowing when it was coming, I set a DataReceived event and loop into the handling method until there were no more bytes to read.
When I attempted this, the loop continued endlessly and blocked my program from other tasks (such as processing the retrieved data) when I continuously received data.
I read about threading in C#, I created a thread that constantly checks for SerialPort.Bytes2Read property so it will know when to retrieve available data.
I created a second thread that can process data while new data is still being read. If bytes have been read and ReadSerial() has more bytes to read and the timeout (restarted every time a new byte is read from the serial) they can still be processed and the frames assembled via a method named DataProcessing() which reads from the same variable being filled by ReadSerial().
This gave me the desired results, but I noticed that with my solution (both ReadSerial() and DataProcessing() threads alive), CPU Usage was skyrocketed all the way to 100%!
How do you approach this problem without causing such high CPU usage?
public static void ReadSerial() //Method that handles Serial Reception
{
while (KeepAlive) // Bool variable used to keep alive the thread. Turned to false
{ // when the program ends.
if (Port.BytesToRead != 0)
{
for (int i = 0; i < 5000; i++)
{
/* I Don't know any other way to
implement a timeout to wait for
additional characters so i took what
i knew from PIC Serial Data Handling. */
if (Port.BytesToRead != 0)
{
RxList.Add(Convert.ToByte(Port.ReadByte()));
i = 0;
if (RxList.Count > 20) // In case the method is stuck still reading
BufferReady = true; // signal the Data Processing thread to
} // work with that chunk of data.
BufferReady = true; // signals the DataProcessing Method to work
} // with the current data in RxList.
}
}
}

I can not understand completely what you are meaning with the "DataReceived" and the "loop". I am also working a lot with Serial Ports as well as other interfaces. In my application I am attaching to the DataReceived Event and also reading based on the Bytes to read, but I dont use a loop there:
int bytesToRead = this._port.BytesToRead;
var data = new byte[bytesToRead];
this._port.BaseStream.Read(data , 0, bytesToRead);
If you are using a loop to read the bytes I recommend something like:
System.Threading.Thread.Sleep(...);
Otherwise the Thread you are using to read the bytes is busy all the time. And this will lead to the fact that other threads cannot be processed or your CPU is at 100%.
But I think you don't have to use a loop for polling for the data if you are using the DataReceived event. If my undertanding is not correct or you need further information please ask.

c# - SerialPort RS-485 and communication limits

I'm trying to communicate to a device using RS-485 through the serial port. Everything works fine, until we're trying to boost the communication to test the speed limit of the card then weird problem seem to occur. We are basically sending a first command with an image as arguments, and then another command to display this image. After every command, the card answers saying that the command was well received. But we are reaching limits too soon and the card is supposed to handle much more.
So I'm wondering since the transmission and the reception are going through the same wire, if there is some sort of collision of data? And should I wait to receive all the data? Is the SerialDataReceivedEventHandler too slow of this situation and should I keep reading the bytes in a while true loop in seperate thread and signal other thread once a complete message is arrived?
Other information :
We already have a protocol for communication : startdelimiter, data,
CRC16, enddelimiter
Sending in 2 commands is the way we do it and cannot be changed.
BaudRate is defined at 115200
The engineer is still working on the program in the card so problem might also be on his end.
English is not my first language so feel free to ask if I was not clear... :)
I recognize SerialPort programming is not my strength, and I've been trying to find some sort of wrapper but I haven't found any that would fit my needs. If someone has one to propose to me that'd be great or maybe someone has an idea of what could be wrong.
Anyway here is a bit of coding :
Thread sending frames :
public void SendOne()
{
timerLast = Stopwatch.GetTimestamp();
while (!Paused && conn.ClientConnState == Connexion.ConnectionState.Connected)
{
timerNow = Stopwatch.GetTimestamp();
if ((timerNow - timerLast) / (double)Stopwatch.Frequency >= 1 / (double)fps)
{
averageFPS.Add((int)((double)Stopwatch.Frequency / (timerNow - timerLast)) + 1);
if (averageFPS.Count > 10) averageFPS.RemoveAt(0);
timerLast = Stopwatch.GetTimestamp();
if (atFrame >= toSend.Count - 1)
{
atFrame = 0;
if (!isLoop)
Paused = true;
}
SendColorImage();
}
}
public void SendColorImage()
{
conn.Write(VIP16.bytesToVIP16(0x70C1, VIP16.Request.SendImage, toSend[++atFrame]));
WaitForResponse();
conn.Write(VIP16.bytesToVIP16(0x70C1, VIP16.Request.DisplayImage, VIP16.DisplayOnArg));
WaitForResponse();
}
private void WaitForResponse()
{
Thread.Sleep(25);
}
So the WaitForResponse() is crucial because if I send another command before the card answered it would go nuts. Although I hate to use Thread.Sleep() because it is not very accurate plus it'd limit my speed to 20fps, and if I use something lower than 25ms, risks of crash is much more likely to occur. So I was about to change the Thread.Sleep to "Read bytes until whole message is received" and ignore the DataReceivedEvent... just wondering if I'm completely off track here?
Tx a lot!
UPDATE 1
First Thank you Brad and 500 - Internal Server Error! But I've decide to stick with the .NET Serial Port for now and improve the Thread.Sleep accuracy (with timebeginperiod). I've decided to wait for the full response to be received and I synchronized my threads like so using ManualResetEventSlim (for speed) :
public static ManualResetEventSlim _waitHandle = new ManualResetEventSlim(false);
Then I changed SendColorIMage to :
public void SendColorImage()
{
conn.Write(VIP16.bytesToVIP16(0x70C1, VIP16.Requetes.SendImage, toSend[++atFrame]));
WaitForResponse();
conn.Write(VIP16.bytesToVIP16(0x70C1, VIP16.Requetes.DisplayImage, VIP16.DisplayOnArg));
WaitForResponse2();
}
private void WaitForResponse()
{
Connexion._waitHandle.Wait(100);
Thread.Sleep(20);
}
private void WaitForResponse2()
{
Connexion._waitHandle.Wait(100);
//Thread.Sleep(5);
}
With SerialDataReceivedEventHandler calling :
public void Recevoir(object sender, SerialDataReceivedEventArgs e)
{
if (!msg.IsIncomplete)
msg = new Vip16Message();
lock (locker)
{
if (sp.BytesToRead > 0)
{
byte[] byteMsg = new byte[sp.BytesToRead];
sp.Read(byteMsg, 0, byteMsg.Length);
msg.Insert(byteMsg);
}
}
if (!msg.IsIncomplete)
{
_waitHandle.Set();
if (MessageRecu != null)
MessageRecu(msg.toByte());
}
}
So I found out that after the second command I didn't need to call Thread.Sleep at all... and after the first one I needed to sleep for at least 20ms for the card not to crash. So I guess it's the time the card needs to receive/process the whole image to it's pixel. AND collision of data shouldn't really occur since I wait until whole message has arrived which means the problem is not on my end! YES! :p

A couple of pointers:
After sending, you'll want to wait for the transfer buffer empty event before reading the response. It's EV_TXEMPTY in unmanaged, I don't recall how it's encapsulated on the managed side - our RS485 code predates the .NET comport component.
You can reprogram the timer chip with a timeBeginPeriod(1) call to get a 1 millisecond resolution on Thread.Sleep().
For what it's worth, we sleep only briefly (1 ms) after send and then enter a reading loop where we keep attempting to read (again, with a 1 ms delay between read attempts) from the port until a full response has been received (or until a timeout or the retry counter is exhausted).
Here's the import declaration for timeBeginPeriod - I don't believe it's directly available in .NET (yet?):
[DllImport("winmm.dll")]
internal static extern uint timeBeginPeriod(uint period);
I hope this helps.

C# serial port problem - too simple to fail, but

Ok, this should be dirt simple. I'm trying to read charactes from a serial device. It's such that if I send a space character, it echos back a string of numbers and EOL. That's it.
I'm using Unity 3.3 (.Net 2.0 support), and the 'serial port' is a Prolific serial-to-USB adaptor. BTW: Using Hyperterminal, it all works perfectly, so I know it's not driver nor hardware.
I can open the port ok. It seems I can send my space with port.Write(" "); But if I even TRY to call ReadChar, ReadByte, or ReadLine (like polling), it freezes up until I unplug the USB, and my console output shows nothing (exceptions were caught).
So instead I set up a DataReceviedHandler, but it's never called.
I've read some posts where people have done just this type of thing with Arduinos etc. (this is not an Arduino but hey), using nothing more than ReadLine. Their code does not work for me (and no answers thus far from those authors).
So, any tips? Do I need to use a different thread? If you know any Unity (Mono) coding, any tips along those lines greatly appreciated.
This code a mashup from http://plikker.com/?p=163 and http://msdn.microsoft.com/en-us/library/system.io.ports.serialport.datareceived.aspx#Y537
using UnityEngine;
using System.Collections;
using System.IO.Ports;
using System;
public class SerialTest : MonoBehaviour {
SerialPort stream;
void Start () {
try {
stream = new SerialPort("COM3", 9600);
stream.Parity = Parity.None;
stream.StopBits = StopBits.One;
stream.DataBits = 8;
stream.Handshake = Handshake.None;
stream.DataReceived += new SerialDataReceivedEventHandler(DataReceviedHandler);
stream.Open();
Debug.Log("opened ok"); // it DOES open ok!
} catch (Exception e){
Debug.Log("Error opening port "+e.ToString()); // I never see this message
}
}
void Update () { // called about 60 times/second
try {
// Read serialinput from COM3
// if this next line is here, it will hang, I don't even see the startup message
Debug.Log(stream.ReadLine());
// Note: I've also tried ReadByte and ReadChar and the same problem, it hangs
} catch (Exception e){
Debug.Log("Error reading input "+e.ToString());
}
}
private static void DataReceviedHandler(
object sender,
SerialDataReceivedEventArgs e)
{
SerialPort sp = (SerialPort)sender; // It never gets here!
string indata = sp.ReadExisting();
Debug.Log("Data Received:");
Debug.Log(indata);
}
void OnGUI() // simple GUI
{
// Create a button that, when pressed, sends the 'ping'
if (GUI.Button (new Rect(10,10,100,20), "Send"))
stream.Write(" ");
}
}

Events are not implemented in Mono SerialPort class, so you won't get any notifications, you have to perform (blocking) read explicitly. Other possible problem - I'm not sure how Unity Behaviours work, are you certain all methods accessing the SerialPort are invoked on the same thread? And you are not disposing you port object, this will also cause problems.

Make sure that you are opening the right port, using correct settings. Here is an example of how you could configure it:
serial = new SerialPort();
serial.ReadBufferSize = 8192;
serial.WriteBufferSize = 128;
serial.PortName = "COM1";
serial.BaudRate = 115200;
serial.Parity = Parity.None;
serial.StopBits = StopBits.One;
// attach handlers
// (appears to be broken in some Mono versions?)
serial.DataReceived += SerialPort_DataReceived;
serial.Disposed += SerialPort_Disposed;
serial.Open();
I recommend the open source RealTerm terminal, it has a rich set of features and can help you debug. Try writing a byte manually using such software, and if it works, then the problem is in your program. Otherwise it might be a driver problem (but more likely it isn't).
[Edit]
Calling SerialPort.ReadLine is actually supposed to block the thread until SerialPort.NewLine is received. Also ReadChar and ReadByte will hang until at least one byte is received. You need to make sure that you are actually receiving characters from the other side, and you won't be receiving them if your app is stuck and cannot send the space.
Since I never used Unity, I am not sure how Update is called, but I am presuming it's fired on a foreground thread in regular intervals (otherwise your app wouldn't freeze).
The example that you linked (Arduino and Unity example) shows that Arduino is sending the data continuously, and that is why their Update method is constantly receiving data (no space character needs to be sent towards the device). If they unplug the device, their app will hang just as well.
Well, maybe not, because in .NET 1.1, default value for ReadTimeout was not infinite, like it is in .NET 2.0.
So, what you can do is:
a. Set the ReadTimeout property to a reasonable value. Default in .NET 2.0 is InfiniteTimeout, which doesn't suit your needs. Cons: your update method will still hang for a while on each call, but not infinitely.
b. Someone said that events are not implemented in MONO SerialPort, so I guess using DataReceived only is not an option.
c. Move your sending logic to the Update method also, so that you don't read data at all, until it's time to read it:
private volatile bool _shouldCommunicate = false;
void Update ()
{
if (_shouldCommunicate) // this is a flag you set in "OnGui"
{
try {
stream.Write(" ");
Debug.Log(stream.ReadLine());
} catch (Exception e){
Debug.Log("Error reading input "+e.ToString());
}
}
}
void OnGUI() // simple GUI
{
if (GUI.Button (new Rect(10,10,100,20), "Send"))
_shouldCommunicate = true;
}
Note that, if your device is not sending data, it will also block at stream.ReadLine(), so make sure your ReadTimeout is set to a reasonable value. You will also want to stop sending at some point, but I leave that to you.
d. Send the space in OnGui like you are doing now, but always check if there is data in your buffer before reading it:
void Update () { // called about 60 times/second
try {
// call our new method
Debug.Log(ReadLineNonBlocking());
} catch (Exception e){
Debug.Log("Error reading input "+e.ToString());
}
}
private StringBuilder sb = new StringBuilder();
string ReadLineNonBlocking()
{
int len = stream.BytesToRead;
if (len == 0)
return "";
// read the buffer
byte[] buffer = new byte[len];
stream.Read(buffer, 0, len);
sb.Append(ASCIIEncoding.ASCII.GetString(buffer));
// got EOL?
if (sb.Length < 2 ||
sb[sb.Length-2] != '\r' ||
sb[sb.Length-1] != '\n')
return "";
// if we are here, we got both EOL chars
string entireLine = sb.ToString();
sb.Length = 0;
return entireLine;
}
Disclaimer: this is directly out of my head, untested, so there may be some syntax errors which I am sure you will handle.

Maybe your problem is the configuration of the serial port. It is important not only to check for BaudRate or StopBits. Also you should configure DTR, RTS, Handshake, everything. This is important cause maybe another program set some ugly values and the configuration must be explicitly set at every start or some settings of the old connection can run you into trouble.
Also maybe take a look into one of these tools:
com0com
Serial Port Monitor
They can help you to stub your serial interface or to take a deeper look into the connection. Also maybe try to talk to your serial device by using HyperTerminal or some similar tool that's proven to work.

Had similar problem with Mono, upgrading to 2.6.7 helped.

Do not mix data event and blocking read. What do you expect to happen if data arrives? That both the read method and the event should get the same received data?
You should also read about:
CTS: http://en.wikipedia.org/wiki/RS-232_RTS/CTS#RTS.2FCTS_handshaking
DTR: http://en.wikipedia.org/wiki/Data_Terminal_Ready
Small serial port tutorial describing all states: http://www.wcscnet.com/Tutorials/SerialComm/Page1.htm

The standard c# System.IO.Pots.SerialPort sucks big time. I suggest to give RJCP.DLL.SerialPortStream library a try. Synchronous read/write is super easy with this delightful library too, no need to jump through the loops with delegate and listeners.