I'm trying to communicate with a piece of hardware over a serial port and I've noticed that it's quite tricky. After reading up on some of the SerialPort caveats, I discovered this post by Ben Voigt on alternative approaches to handle serial communication in .NET with the SerialPort class.
However, this code is quite old (8 years or so as of today) and the Stream class now exposes asynchronous read/write support, so I was wondering how the code in that blog could be re-written to account for the following scenarios:
The device periodically sends a status update through the port (auto generated)
I can send a byte array that represents a command that doesn't need a response
I can send a byte array that represents a command that does need a response, and the device will send that reply data through the port
So far, I have a simple read loop that looks a bit like this:
public async IAsyncEnumerable<byte> ReadBytesAsync()
{
while (_port.IsOpen)
{
byte[]? buffer = new byte[1];
try
{
await _port.BaseStream.ReadAsync(buffer, 0, 1);
}
catch (Exception e)
{
// Do stuff
}
if (buffer is not null) yield return buffer[0];
await Task.Delay(1); // Wait a bit before reading again
}
}
I also have a GetRepliesAsync() method that await foreaches over the read bytes and constructs the right data types for the replies.
The autogenerated status update gets read correctly when received, but if I use _port.BaseStream.WriteAsync(commandBytes, 0, commandBytes.Length), the device receives the update but I somehow don't receive its reply through the base stream.
What could I be doing wrong, and how can I actually read the received bytes?
UPDATES
1. Modified read and added delay
I got the code to work somewhat with the following approach:
while (_port.IsOpen)
{
if (cancellationToken.IsCancellationRequested) yield break;
int bufferLength = 2048; // Arbitrary buffer length
byte[] buffer = new byte[bufferLength];
try
{
// Partially using Ben Voigt's approach
int bytesRead = await _port.BaseStream
.ReadAsync(buffer.AsMemory(0, bufferLength), cancellationToken);
nextBytes = new byte[bytesRead];
Buffer.BlockCopy(buffer, 0, nextBytes, 0, bytesRead);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
Console.WriteLine("Terminating serial read polling...");
yield break;
}
if (nextBytes is not null)
{
for (int i = 0; i < nextBytes.Length; i++)
{
yield return nextBytes[i];
}
}
// Introduce delay in case a write operation is going on
await Task.Delay(200, cancellationToken);
}
But I still don't see a way to guarantee that each write will happen during the 200 millisecond delay.
Related
I'm currently using the ReadAsync method in NetworkStream, and thus far its been pretty stable. The problem occurs when the data received from the server is larger than the buffer itself.
ReadCallBack assumes that the buffer contains a complete and valid message, as a new buffer is created on every call to BeginAsyncRead. Thus, when the response is too large for one buffer, it gets split between multiple reads and thus multiple buffers.
My ReadCallBack then assumes each buffer is an independent message (instead of adding them together) and ends up failing the deserialization on both.
My question is, what is the correct way to handle messages that are bigger than the size of the buffer, specially if you want to do the reads asynchronously?
A character itself could be split between two buffers so this gets quite tricky.
Below is my code for ReadAsync and ReadCallBack.
public async Task ReadForever() {
while(this.IsConnected) {
await BeginAsyncRead();
}
}
public async Task BeginAsyncRead() {
if (this.Stream.CanRead) {
try {
ReceiverState readState = new ReceiverState(this.Stream);
var task = this.Stream.ReadAsync(readState.buffer, 0, ReceiverState.bufferSize);
await task.ContinueWith(bytesRead => ReadCallBack(bytesRead.Result, readState));
}
catch (Exception e) {
Console.WriteLine("Error");
}
} else {
Console.WriteLine("Unreadable stream");
}
}
private void ReadCallBack(int numBytesRead, ReceiverState state) {
if (numBytesRead > 0) {
string payload = Encoding.ASCII.GetString(state.buffer, 0, numBytesRead);
string[] responses = payload.Split(new string[] {"\n"}, StringSplitOptions.RemoveEmptyEntries);
foreach (string res in responses){
var t = Task.Run(() => this.OnRead(res));
}
} else {
Console.WriteLine("Corrupted number of bytes received");
}
}
Note : All responses sent by the server contain a newline at the end.
Note : ReadForever is called inside Task.Run, my application can receive messages from the server as notifications, thus I must always be reading for incoming messages (I don't know when a notification may arrive).
I have written some code to get a webpage through a proxy using sockets. In essence, it works but reading the response has some strange behavior that is really tripping me up.
When I go to read the response after sending the GET command it is 0 bytes. It takes a few ticks before there is data to read. I don't want to hard code a delay in here as I am trying to write performant reliable code so I have coded a while loop that keeps reading the response until it more than 0.
This works for the first chunk but trying to read subsequent chunks is a problem. If i instantly try to read the response it will be 0 bytes so I need to check the subsequent reads also if they are greater than 0.
So to read the whole response I tried to check if the response is equal to the size of the buffer. If it is equal to the size of the buffer then I carry on and try to read another chunk. This has a few issues also. Sometimes the response will read less than the size of the buffer but there is still more to come, i guess I am reading it faster than they are sending it because if I add a Thread.Sleep() then the buffer will always be full but I don't think it is good practice to hardcode this because I don't know how fast they will be sending. This code will be used for multiple things and will be running on hundreds of threads so performance is everything.
Also if the last chunk just happens to be the size of my buffer then I think the loop will lock, This whole approach I have taken is horrible but I can't see how I should be reading it. I have seen the asynchronous examples but I think that will add to the overall complexity of my code as I just have 1 set process which I will run in many threads.
How do I efficiently read the response when I can't guarantee the next chunk will have data or be full even if there is more data to come?
Sorry for long text but I wanted to explain my thinking. Here is my code:
// Data buffer for incoming data.
byte[] bytes = new byte[1024];
// Connect to a remote device.
try
{
var proxyIpAddress = IPAddress.Parse("123.123.123.123"); //omitted
IPEndPoint remoteEP = new IPEndPoint(proxyIpAddress, 60099);
// Create a TCP/IP socket.
Socket sender = new Socket(proxyIpAddress.AddressFamily,
SocketType.Stream, ProtocolType.Tcp);
// Connect the socket to the remote endpoint. Catch any errors.
try
{
sender.Connect(remoteEP);
Console.WriteLine("Socket connected to {0}",
sender.RemoteEndPoint.ToString());
sender.Send(Encoding.ASCII.GetBytes($"CONNECT google.com:80 HTTP/1.0\r\n\r\n"));
int bytesRec = 0;
while (bytesRec == 0)
{
// Receive the response from the remote device.
bytesRec = sender.Receive(bytes);
Console.WriteLine("{0}",
Encoding.ASCII.GetString(bytes, 0, bytesRec));
}
//clear buffer
bytes = new byte[1024];
bytesRec = 0;
sender.Send(Encoding.ASCII.GetBytes("GET / HTTP/1.0\r\n\r\n"));
//wait for response
while (bytesRec == 0) //if i dont add this it returns before it actually gets data
{
// Receive the response from the remote device.
bytesRec = sender.Receive(bytes);
Console.WriteLine("{0}",
Encoding.ASCII.GetString(bytes, 0, bytesRec));
}
if(bytes.Length == bytesRec) //full buffer so likely more but maybe not if final packet exactly 1024?
{
while (bytes.Length == bytesRec) //again if i miss this it returns too early
{
int subsequentBytes = 0;
while(subsequentBytes == 0) //this can get stuck if last packet exactly size of buffer i think
{
subsequentBytes = sender.Receive(bytes);
Console.WriteLine("{0}",
Encoding.ASCII.GetString(bytes, 0, subsequentBytes));
//this doesn't work. even when there are subsequent bytes sometimes it reads less
//than the size of the buffer so it exits prematurely. If I add a Thread.Sleep() here
// then it works but I don't want to hardcode the delay. How do I read this buffer properly?
Thread.Sleep(1000);
if (subsequentBytes > 0) bytesRec = subsequentBytes;
}
}
}
// Release the socket.
sender.Shutdown(SocketShutdown.Both);
sender.Close();
}
catch (Exception e)
{
Console.WriteLine("Unexpected exception : {0}", e.ToString());
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
I understand this is difficult to follow and a lot of writing so if anyone perseveres with this they have my gratitude as the only option I can see is hardcoded pauses which will hurt performance and may still have issues.
EDIT
I have done some experiementing with different servers. If I ping the server then set a Thread.Sleep(pingValue) it works fine but if i set the sleep to lower than ping i get same issue.
Is there some good way with the .net libraries to account for this latency so I am not under/overestimating?
I've been working with windows app store programming in c# recently, and I've come across a problem with sockets.
I need to be able to read data with an unknown length from a DataReader().
It sounds simple enough, but I've not been able to find a solution after a few days of searching.
Here's my current receiving code (A little sloppy, need to clean it up after I find a solution to this problem. And yes, a bit of this is from the Microsoft example)
DataReader reader = new DataReader(args.Socket.InputStream);
try
{
while (true)
{
// Read first 4 bytes (length of the subsequent string).
uint sizeFieldCount = await reader.LoadAsync(sizeof(uint));
if (sizeFieldCount != sizeof(uint))
{
// The underlying socket was closed before we were able to read the whole data.
return;
}
reader.InputStreamOptions
// Read the string.
uint stringLength = reader.ReadUInt32();
uint actualStringLength = await reader.LoadAsync(stringLength);
if (stringLength != actualStringLength)
{
// The underlying socket was closed before we were able to read the whole data.
return;
}
// Display the string on the screen. The event is invoked on a non-UI thread, so we need to marshal
// the text back to the UI thread.
//MessageBox.Show("Received data: " + reader.ReadString(actualStringLength));
MessageBox.updateList(reader.ReadString(actualStringLength));
}
}
catch (Exception exception)
{
// If this is an unknown status it means that the error is fatal and retry will likely fail.
if (SocketError.GetStatus(exception.HResult) == SocketErrorStatus.Unknown)
{
throw;
}
MessageBox.Show("Read stream failed with error: " + exception.Message);
}
You are going down the right lines - read the first INT to find out how many bytes are to be sent.
Franky Boyle is correct - without a signalling mechanism it is impossible to ever know the length of a stream. Thats why it is called a stream!
NO socket implementation (including the WinSock) will ever be clever enough to know when a client has finished sending data. The client could be having a cup of tea half way through sending the data!
Your server and its sockets will never know! What are they going to do? Wait forever? I suppose they could wait until the client had 'closed' the connection? But your client could have had a blue screen and the server will never get that TCP close packet, it will just be sitting there thinking it is getting more data one day?
I have never used a DataReader - i have never even heard of that class! Use NetworkStream instead.
From my memory I have written code like this in the past. I am just typing, no checking of syntax.
using(MemoryStream recievedData = new MemoryStream())
{
using(NetworkStream networkStream = new NetworkStream(connectedSocket))
{
int totalBytesToRead = networkStream.ReadByte();
// This is your mechanism to find out how many bytes
// the client wants to send.
byte[] readBuffer = new byte[1024]; // Up to you the length!
int totalBytesRead = 0;
int bytesReadInThisTcpWindow = 0;
// The length of the TCP window of the client is usually
// the number of bytes that will be pushed through
// to your server in one SOCKET.READ method call.
// For example, if the clients TCP window was 777 bytes, a:
// int bytesRead =
// networkStream.Read(readBuffer, 0, int.Max);
// bytesRead would be 777.
// If they were sending a large file, you would have to make
// it up from the many 777s.
// If it were a small file under 777 bytes, your bytesRead
// would be the total small length of say 500.
while
(
(
bytesReadInThisTcpWindow =
networkStream.Read(readBuffer, 0, readBuffer.Length)
) > 0
)
// If the bytesReadInThisTcpWindow = 0 then the client
// has disconnected or failed to send the promised number
// of bytes in your Windows server internals dictated timeout
// (important to kill it here to stop lots of waiting
// threads killing your server)
{
recievedData.Write(readBuffer, 0, bytesReadInThisTcpWindow);
totalBytesToRead = totalBytesToRead + bytesReadInThisTcpWindow;
}
if(totalBytesToRead == totalBytesToRead)
{
// We have our data!
}
}
}
I am trying to write network part for my game in C# using System.Net.Sockets and TcpClient class.
Each update server is sending information to client.
All information is built into 2kb packets, so in 1 update 1-2-3-5-10 packets can be sent.
Client is checking information and if information has right format - then reading it.
Everything is working fine, until server starts trying to send too many packets.
When it happens client time to time is receiving packets with wrong data 1 of 20-50 packets usually.
For example, 1-2 packets for 1 update usually are received fine, 3-10 packets for update giving wrong data streams.
If I am starting several clients in 1 time, that should get same data streams from server - they get different numbers of success and fail data streams.
What am I doing wrong, and how can I evade this wrong data streams?
Am I just sending too much data in 1 ms and it is needed to send it over time?
This is the sending information:
TcpClient client;
public void SendData(byte[] b)
{
//Try to send the data. If an exception is thrown, disconnect the client
try
{
lock (client.GetStream())
{
client.GetStream().BeginWrite(b, 0, b.Length, null, null);
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
This is the receiving information:
byte[] readBuffer;
int byfferSize = 2048;
private void StartListening()
{
client.GetStream().BeginRead(readBuffer, 0, bufferSize, StreamReceived, null);
}
private void StreamReceived(IAsyncResult ar)
{
int bytesRead = 0;
try
{
lock (client.GetStream())
{
bytesRead = client.GetStream().EndRead(ar); // просмотр длины сообщения
}
}
catch (Exception ex)
{ MessageBox.Show(ex.Message); }
//An error happened that created bad data
if (bytesRead == 0)
{
Disconnect();
return;
}
//Create the byte array with the number of bytes read
byte[] data = new byte[bytesRead];
//Populate the array
for (int i = 0; i < bytesRead; i++)
data[i] = readBuffer[i];
//Listen for new data
StartListening();
//Call all delegates
if (DataReceived != null)
DataReceived(this, data);
}
It is main network code.
I don't know what you do with the data after you've received it, but it's quite possible that you're not reading all of the data from the connection. You have:
bytesRead = client.GetStream().EndRead(ar);
There's no guarantee that the number of bytes you've read are all of the bytes that the server sent. For example, the server could have sent 2,048 bytes, but when you called Read, there were only 1,024 bytes available. The rest of them are still "in transit." As the documentation for NetworkStream.Read says:
The Read operation reads as much data as is available, up to the number of bytes specified by the size parameter
You could be getting partial packets. If your DataReceived handlers assume that the data buffer contains a complete packet, then you're going to have problems.
To reliably read from a network stream, you need to know how much data you're supposed to read, or you need a record separator. Something has to make sure that if you're expecting a complete packet that you get a complete packet before you try to process it. Your code just checks to see if bytesRead is not 0. If it's anything else, you just pass it on. This is going to be a problem unless your DataReceived handlers know how to buffer partial packets.
On another note, you really don't need to lock the network stream. Unless you can have several threads reading from the same stream. And that would be disastrous. Ditch the lock. You don't need it.
hey I'm writing on an Server-Client program
but when my client sends something, it never reaches my server!
I'm sending like this:
public void Send(string s)
{
char[] chars = s.ToCharArray();
byte[] bytes = chars.CharToByte();
nstream.Write(bytes, 0, bytes.Length);
nstream.Flush();
}
and Receiving in a background thread like this
void CheckIncoming(object dd)
{
RecievedDelegate d = (RecievedDelegate)dd;
try
{
while (true)
{
List<byte> bytelist = new List<byte>();
System.Threading.Thread.Sleep(1000);
int ssss;
ssss = nstream.ReadByte();
if (ssss > 1)
{
System.Diagnostics.Debugger.Break();
}
if (bytelist.Count != 0)
{
d.Invoke(bytelist.ToArray());
}
}
}
catch (Exception exp)
{
MSGBOX("ERROR:\n" + exp.Message);
}
}
the ssss int is never > 1
whats happening here???
NetworkStream.Flush() actually has no effect:
The Flush method implements the Stream..::.Flush method; however, because NetworkStream is not buffered, it has no affect [sic] on network streams. Calling the Flush method does not throw an exception
How much data is being sent?
If you don't send enough data it may remain buffered at the network level, until you close the stream or write more data.
See the TcpClient.NoDelay property for a way to disable this, if you are only going to be sending small chunks of data and require low latency.
You should change the check of the return value to if (ssss >= 0).
ReadByte returns a value greater or equal 0 if it succeeds to read a byte (source).
To elaborate on Marc's comment: How is nstream created? Maybe there is an underlying class that does not flush.
well, Im creating a TcpClient, and use GetStream(); to get the NetworkStream