We Keep Coding

Endless while loop while polling data out of received bytes

I use some third party code from which i thought it worked...it is a loop which is polling some data out of the received Socket-bytes with a ByteBuffer and has to react to the timeout in case when the NetworkTpdu is null, but it doesn't! The while-loop is endless...Somebody an idea?
int start = Environment.TickCount;
int myTimeout = 5000;
while (NetworkTpdu.CreateFromBuffer(_receiveBuffer, false) == null && (!_masterMode || Environment.TickCount - start < myTimeout))
{
_receiveBuffer.WaitForByte(myTimeout, false);
}

it will probably be easier to find the problem if you split the method into separate statements. For example:
var sw = Stopwatch.StartNew();
var timeout = TimeSpan.FromSeconds(5);
while (true)
{
var buffer = NetworkTpdu.CreateFromBuffer(_receiveBuffer, false);
if (buffer != null)
{
Console.WriteLine("Buffer is not null");
return;
}
if (_masterMode)
{
if (sw.Elapsed > timeout)
{
Console.WriteLine("Timeout!");
return;
}
}
Console.WriteLine("Wait for byte: " + sw.ElapsedMilliseconds);
_receiveBuffer.WaitForByte(timeout.TotalMilliseconds, false);
// Check if WaitForByte succeeded or timed out, and handle the result
}
This should make it easy to find the problem:
Wait for Byte is not printed -> CreateFromBuffer never returns
Wait for Byte is only printed once -> WaitForByte never returns
Wait for Byte is printed with a value over 5000 -> _masterMode is false
Edit: As canton7 mentioned in the comments, you can do the same thing with a debugger by placing breakpoints or tracepoints at appropriate places. This is merely a way to make the execution order more apparent.

COM: Device only sends data when things are changing. How to deal with that?

So I want to connect to a device via Serial that only sends data when things are changing with the settings on the device (a measuring device).
I use C# and .Net's SerialPort.
I am able to read data and it looks kind of good. But there are a few problems I encountered.
I realized my reading- method (ReadExistingData()) so that it will constantly use the SerialDataReceivedEventHandler when there's new data.
Unfortunately, when I read it like that (probably because of varying package sizes) it will read very chaotically and thus I need to "catch" the first initiating byte (here it's 0xA5).
That means, I always check whether the byte I just received is a 0xA5 and if it is, I read the rest.
But I feel like that way I am missing some commands my device sends me and thats why I cant display the data consistently right, only from time to time.
On a side note: The device sends the device time and a value. The value is delayed and kind of unaccurate, but the time is always right on spot. The other parameters it sends are always weirded out and dont seem to be recognized and thus changed at all.
To display data I use the console for testing purposes, and the whole construct seems to be very reactive to Console outputs.
Here's a little code snippet:
class Device
{
private int stdMessageLengthInBytes = 5;
public DeviceData processedData;
byte[] dataBuffer;
int receivedMessages = 0;
public Device()
{
// Initialize BaseClass (SerialPort derivative)
this.port = new System.IO.Ports.SerialPort();
// Initialize Device
this.data = new byte[stdMessageLengthInBytes];
this.processedData = new P8005_Data();
dataBuffer = new byte[stdMessageLengthInBytes];
}
// Is supposed to read the data from serial port
protected override void ReadExistingData()
{
// Check whether buffer is empty -> Try to catch a 0xA5
if (dataBuffer[0] == 0x00)
{
port.Read(dataBuffer, 0, 1);
}
// If no 0xA5 was catched, restart
if (dataBuffer[0] != 0xA5)
{
dataBuffer = new byte[stdMessageLengthInBytes]; // Reset buffer
return;
}
// Read next byte -> Command byte
port.Read(dataBuffer, 1, 1);
// If command was empty, restart
if (dataBuffer[1] == 0x00)
{
dataBuffer = new byte[stdMessageLengthInBytes]; // Reset buffer
return;
}
// If its time that is communicated: Read 3 bytes
if (dataBuffer[1] == 0x06)
{
// 4 ms delay seems to be needed otherwise it wont function correctly somehow
System.Threading.Thread.Sleep(5);
port.Read(dataBuffer, 2, 3);
// Write buffer to actual raw data byte array
this.data = dataBuffer;
dataBuffer = new byte[stdMessageLengthInBytes]; // Reset buffer
}
// Otherwise: Just read 2 bytes
System.Threading.Thread.Sleep(5); // Needed delay
port.Read(dataBuffer, 2, 2);
// Write buffer to actual raw data byte array
this.data = dataBuffer;
dataBuffer = new byte[stdMessageLengthInBytes]; // Reset buffer
}
// Method called by SerialDataReceivedEventHandler
protected override void DataReceivedOnComPort(object sender, SerialDataReceivedEventArgs e)
{
bool valid = false;
ReadExistingData(); // Read data from COM- Port
lock (processedData)
{
switch (data[1]) // Check command byte
{
// Time (3 btyes)
case (0x06):
processedData.currentTime = String.Format("{0:D2}:{1:D2}:{2:D2}", DecodeBcd(data[2]), DecodeBcd(data[3]), DecodeBcd(data[4]));
valid = true;
receivedMessages++;
break;
// Value (2 bytes)
case (0x0D):
double val = 0;
val += DecodeBcd(data[2]) * 100;
val += DecodeBcd(data[3]);
val /= 10;
processedData.currentValue = val;
valid = true;
receivedMessages++;
break;
// ... here are various other hex- code that represent a command from the device (2 btyes)
default:
valid = false;
break;
}
}
// only to check when
if (valid)
{
Console.WriteLine("Received Valid Messages: {0}", receivedMessages);
ConsoleOutput();
}
}
}
On a note: The initialization of the port happens in another method from the base class and works fine.
Is there anything I am missing? How to deal with something like that? Are there any improvements that would help improving my performance? I thought about threading with locks, but I dont think that is the solution somehow... Or maybe everything is just a console problem?
EDIT:
I know changed my code (as #jdweng suggested) so that I put everything in a buffer (basically List<byte> mainBuffer. Then, I take all bytes in the buffer whenever its possbile and work with them, skimming it for 0xA5. When one is found, I read the command and determine how long the "message" has to be according to it (Time -> +3 bytes, Data -> +2 bytes, Other -> +1 byte). Then I can work off those messages (I put them into a List<byte[]>) and determine my output to my screen.
However, even after outsourcing the chopping up into messages and processing the messages, I still seem to either miss some messages, since some action are just not registered and have a big delay, or my processing is wrong. What I can think of is that because I lock my mainBuffer maybe some data isnt written to it.
Is this really this time critical? There is a software that comes with the device and it doesnt seem to have such big problems with delay and slightly wrong values...

Since you don't have the exact specs and/or an unreliable connection (which with serial data has to be expected) you need to sync to the 0xa5 at every message. I would just run every single byte you receive through a parser while keeping the state of the currently received message.
Make sure you validate your input since there are a bunch of things that can go wrong if you get messed up serial data. For example if there is an 0xa5 in the other message types, you might miss your next message. To prevent that I strongly recommend to either get to the specs if possible or code more logic based on data observations.
private const int MESSAGE_LENGTH = 5;
private const int VALUE_COMMAND = 0x0D;
private const int VALUE_SIZE = 4;
private const int TIME_COMMAND = 0x06;
private const int TIME_SIZE = 5;
private byte[] _message = new byte[MESSAGE_LENGTH];
private int _messagePos = 0;
private void port_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
var data = new byte[_serialPort.BytesToRead];
_serialPort.Read(data, 0, data.Length);
foreach (var b in data)
{
_message[_messagePos] = b;
if (_messagePos == 0 && b != 0xa5)
continue;
++_messagePos;
if (_messagePos > 2) // if command byte present, process command of any size
ProcessCommand(_message[1]);
}
}
private void ProcessCommand(byte command)
{
if (_messagePos == VALUE_SIZE && command == VALUE_COMMAND)
{
// parse value...
_messagePos = 0;
}
else if (_messagePos == TIME_SIZE && _message[1] == TIME_COMMAND)
{
// parse time...
_messagePos = 0;
}
}

SerialPort.ReadLine() slow compared to manual method?

I've recently implemented a small program which reads data coming from a sensor and plotting it as diagram.
The data comes in as chunks of 5 bytes, roughly every 500 µs (baudrate: 500000). Around 3000 chunks make up a complete line. So the total transmission time is around 1.5 s.
As I was looking at the live diagram I noticed a severe lag between what is shown and what is currently measured. Investigating, it all boiled down to:
SerialPort.ReadLine();
It takes around 0.5 s more than the line to be transmitted. So each line read takes around 2 s. Interestingly no data is lost, it just lags behind even more with each new line read. This is very irritating for the user, so I couldn't leave it like that.
I've implemented my own variant and it shows a consistent time of around 1.5 s, and no lag occurs. I'm not really proud of my implementation (more or less polling the BaseStream) and I'm wondering if there is a way to speed up the ReadLine function of the SerialPort class. With my implementation I'm also getting some corrupted lines, and haven't found the exact issue yet.
I've tried changing the ReadTimeout to 1600, but that just produced a TimeoutException. Although the data arrived.
Any explanation as of why it is slow or a way to fix it is appreciated.
As a side-note: I've tried this on a Console application with only SerialPort.ReadLine() as well and the result is the same, so I'm ruling out my own application affecting the SerialPort.
I'm not sure this is relevant, but my implementation looks like this:
LineSplitter lineSplitter = new LineSplitter();
async Task<string> SerialReadLineAsync(SerialPort serialPort)
{
byte[] buffer = new byte[5];
string ret = string.Empty;
while (true)
{
try
{
int bytesRead = await serialPort.BaseStream.ReadAsync(buffer, 0, buffer.Length).ConfigureAwait(false);
byte[] line = lineSplitter.OnIncomingBinaryBlock(this, buffer, bytesRead);
if (null != line)
{
return Encoding.ASCII.GetString(line).TrimEnd('\r', '\n');
}
}
catch
{
return string.Empty;
}
}
}
With LineSplitter being the following:
class LineSplitter
{
// based on: http://www.sparxeng.com/blog/software/reading-lines-serial-port
public byte Delimiter = (byte)'\n';
byte[] leftover;
public byte[] OnIncomingBinaryBlock(object sender, byte[] buffer, int bytesInBuffer)
{
leftover = ConcatArray(leftover, buffer, 0, bytesInBuffer);
int newLineIndex = Array.IndexOf(leftover, Delimiter);
if (newLineIndex >= 0)
{
byte[] result = new byte[newLineIndex+1];
Array.Copy(leftover, result, result.Length);
byte[] newLeftover = new byte[leftover.Length - result.Length];
Array.Copy(leftover, newLineIndex + 1, newLeftover, 0, newLeftover.Length);
leftover = newLeftover;
return result;
}
return null;
}
static byte[] ConcatArray(byte[] head, byte[] tail, int tailOffset, int tailCount)
{
byte[] result;
if (head == null)
{
result = new byte[tailCount];
Array.Copy(tail, tailOffset, result, 0, tailCount);
}
else
{
result = new byte[head.Length + tailCount];
head.CopyTo(result, 0);
Array.Copy(tail, tailOffset, result, head.Length, tailCount);
}
return result;
}
}

I ran into this issue in 2008 talking to GPS modules. Essentially the blocking functions are flaky and the solution is to use APM.
Here are the gory details in another Stack Overflow answer: How to do robust SerialPort programming with .NET / C#?
You may also find this of interest: How to kill off a pending APM operation

NetworkStream Equivelant

I have been trying to create a metro application but there is a problem: StreamSocket doesn't really do what I want to do (I think)
Here is an excerpt my code from .Net that works:
try
{
TCP = new TcpClient(server, port);
Stream = TCP.GetStream();
Read = new StreamReader(Stream);
Write = new StreamWriter(Stream);
}
catch (Exception e)
{
Console.WriteLine("Error connecting to " + server + ": " + e);
return;
}
// Identify
Write.WriteLine("LOGIN " + Username);
Write.Flush();
while (Connected)
{
try
{
if ((line = Read.ReadLine()) != null && Connected)
I can't get StreamSocket to work... it requires you to know the length of the string that's coming in and I don't know what it will be - it varies. Is there any way to do this that will work?
This is what I have but it doesn't work:
try
{
// Connect to the server (in our case the listener we created in previous step).
await Socket.ConnectAsync(new HostName("example.com"), "1111");
}
catch (Exception exception)
{
// If this is an unknown status it means that the error is fatal and retry will likely fail.
if (SocketError.GetStatus(exception.HResult) == SocketErrorStatus.Unknown)
{
throw;
}
}
// Create a DataWriter if we did not create one yet. Otherwise use one that is already cached.
Writer = new DataWriter(Socket.OutputStream);
Listener = new DataReader(Socket.InputStream);
Debug.WriteLine(Socket.Information.RemoteAddress.CanonicalName); //Check if IP is correct
SendRaw("LOGIN " + Nickname);
string line = "";
Connected = true;
while (Connected)
{
if (Listener.UnconsumedBufferLength != 0)
{
line = Listener.ReadString(Listener.UnconsumedBufferLength);
Debug.WriteLine(line);
}
}
}
async public void SendRaw(string str)
{
Writer.WriteString(str);
// Write the locally buffered data to the network.
try
{
await Writer.StoreAsync();
}
catch (Exception exception)
{
// If this is an unknown status it means that the error if fatal and retry will likely fail.
if (SocketError.GetStatus(exception.HResult) == SocketErrorStatus.Unknown)
{
throw;
}
}
}
Any help would be appreciated!

First things first: your original code is a DOS attack waiting to happen. If possible, I would recommend changing the protocol to include a length prefix before every string so you can tell how big it will be before allocating memory for it.
Second things second: the DataReader class must read a number of bytes into its internal buffer before it can interpret them. You read into this buffer by calling LoadAsync.
However, if you want to read a string of arbitrary length, you'll have to read into a buffer and scan for your newline yourself, resizing the buffer (or adding new buffers) as necessary if the newline isn't found, up to some maximum size.
Update:
Set InputStreamOptions to Partial; the you can call LoadAsync with an arbitrary large buffer size (e.g. 1024). After getting data, call ReadString(UnconsumedBufferLength). Each time you do this, you may get part of a line, a line, or more than a line. So you'll have to build up a string and then Split by \n, keeping any partial line at the end for the next time through the loop.

networkstream always empty!

hey I'm writing on an Server-Client program
but when my client sends something, it never reaches my server!
I'm sending like this:
public void Send(string s)
{
char[] chars = s.ToCharArray();
byte[] bytes = chars.CharToByte();
nstream.Write(bytes, 0, bytes.Length);
nstream.Flush();
}
and Receiving in a background thread like this
void CheckIncoming(object dd)
{
RecievedDelegate d = (RecievedDelegate)dd;
try
{
while (true)
{
List<byte> bytelist = new List<byte>();
System.Threading.Thread.Sleep(1000);
int ssss;
ssss = nstream.ReadByte();
if (ssss > 1)
{
System.Diagnostics.Debugger.Break();
}
if (bytelist.Count != 0)
{
d.Invoke(bytelist.ToArray());
}
}
}
catch (Exception exp)
{
MSGBOX("ERROR:\n" + exp.Message);
}
}
the ssss int is never > 1
whats happening here???

NetworkStream.Flush() actually has no effect:
The Flush method implements the Stream..::.Flush method; however, because NetworkStream is not buffered, it has no affect [sic] on network streams. Calling the Flush method does not throw an exception
How much data is being sent?
If you don't send enough data it may remain buffered at the network level, until you close the stream or write more data.
See the TcpClient.NoDelay property for a way to disable this, if you are only going to be sending small chunks of data and require low latency.

You should change the check of the return value to if (ssss >= 0).
ReadByte returns a value greater or equal 0 if it succeeds to read a byte (source).
To elaborate on Marc's comment: How is nstream created? Maybe there is an underlying class that does not flush.

well, Im creating a TcpClient, and use GetStream(); to get the NetworkStream