I have a race condition or something like it. I mean if I toggle a breakpoint before reading from COM, everything is good. But when i'm toggling it off, it freezes. writing:
public void Send(ComMessage message)
{
byte[] bytes = message.Serialise();
if (!_outputPort.IsOpen)
_outputPort.Open();
try
{
byte[] size = BitConverter.GetBytes(bytes.Length);
_outputPort.Write(size, 0, size.Length);
_outputPort.Write(bytes, 0, bytes.Length);
}
finally
{
if (_outputPort != _inputPort)
_outputPort.Close();
}
}
reading
private void InputPortOnDataReceived(object sender, SerialDataReceivedEventArgs serialDataReceivedEventArgs)
{
var port = (SerialPort) sender;
byte[] sizeBuffer = new byte[sizeof(long)];
port.Read(sizeBuffer, 0, sizeBuffer.Length);
int length = BitConverter.ToInt32(sizeBuffer, 0);
byte[] buffer = new byte[length];
int i = 0;
while (i < length)
{
int readed = port.Read(buffer, i, length - i);
i += readed;
}
var message = ComMessage.Deserialize(buffer);
MessageReceived(this, message);
}
for example, message has 625 bytes length. If I toggle a breakpoint, port.BytesToRead is equal 625, but if I disable it, byte count is 621.
Strange, but it works for a little amount of bytes (for short messages), but doesn't for long.
Please, advice.
message has 625 bytes length. If I toggle a breakpoint,
port.BytesToRead is equal 625, but if I disable it, byte count is 621.
You never check the first Read to see how many bytes it read. It may have read less than sizeof(long) bytes in. However, that is not the source of your problem, your main problem is you are making a buffer of size long but long is Int64, you are calling ToInt32 (and writing a Int32 in your sender).
The reason your byte count is 261 instead of 265 is because the first 4 bytes of your message is sitting in sizeBuffer[4] through sizeBuffer[7] which you never processed.
To fix this you should be either doing sizeof(int) or even better to make it more obvious that the buffer is for the ToInt32 call, use sizeof(Int32)
Related
My code is designed to get data from a serial device and print its contents to a MS Forms Application. The IDE i use is Visual Studio 2019 - Community.
The device does send a variable size packet. First i have to decode the packet "header" to get crucial information for further processing which is the first packet channel as well as the packet length.
Since the packet does neither contain a line ending, nor a fixed character at the end, the functions SerialPort.ReadTo() and SerialPort.ReadLine() are not useful. Therefore only SerialPort.Read(buf,offset,count) can be used
Since sending rather large packets (512bytes) does take time, I've implemented a function for calculation of a desired wait time, which is defined as (1000ms/baud-rate*(8*byte-count))+100ms
While testing, I've experienced delay, much more than the desired wait times, so implemented a measure function for different parts of the function.
In regular cases (with desired wait times) i except a log to console like this:
Load Header(+122ms) Load Data (+326ms) Transform (+3ms)
But its only like this for a few variable amount of records, usually 10, after that, the execution times are much worse:
Load Header(+972ms) Load Data (+990ms) Transform (+2ms)
Here you can see the complete function:
private void decodeWriteResponse(int identifier, object sender, SerialDataReceivedEventArgs e)
{
/* MEASURE TIME NOW */
long start = new DateTimeOffset(DateTime.UtcNow).ToUnixTimeMilliseconds();
var serialPort = (SerialPort)sender; //new serial port object
int delay = ComPort.getWaitTime(7); //Returns the wait time (1s/baudrate * bytecount *8) +100ms Additional
Task.Delay(delay).Wait(); //wait until the device has send all its data!
byte[] databytes = new byte[6]; //new buffer
try
{
serialPort.Read(databytes, 0, 6); //read the data
}
catch (Exception) { };
/* MEASURE TIME NOW */
long between_header = new DateTimeOffset(DateTime.UtcNow).ToUnixTimeMilliseconds();
/* Read the Data from Port */
int rec_len = databytes[1] | databytes[2] << 8; //Extract number of channels
int start_chnl = databytes[3] | databytes[4] << 8; //Extract the first channel
delay = ComPort.getWaitTime(rec_len+7); //get wait time
Task.Delay(delay).Wait(); //wait until the device has send all its data!
byte[] buf = new byte[rec_len-3]; //new buffer
try
{
serialPort.Read(buf, 0, rec_len-3); //read the data
}
catch (Exception) {}
/* MEASURE TIME NOW */
long after_load = new DateTimeOffset(DateTime.UtcNow).ToUnixTimeMilliseconds();
/* Now perform spectrum analysis */
decodeSpectrumData(buf, start_chnl, rec_len-4);
/*MEASURE TIME NOW */
long end = new DateTimeOffset(DateTime.UtcNow).ToUnixTimeMilliseconds();
Form.rtxtDataArea.AppendText("Load Header(+" + (between_header - start).ToString() + "ms) Load Data (+" + (after_load - between_header).ToString() + "ms) Transform (+" + (end - after_load) + "ms)\n");
/*Update the Write handler */
loadSpectrumHandler(1);
}
What could cause this issue?
I already tested this with "debug" in Visual Studio, and as "Release" standalone, but there is no difference.
Instead of trying to figure out how long a message will take to arrive at the port, why not just read the data in a loop until you have it all? For example, read the header and calculate the msg size. Then read that number of bytes. Ex:
// See if there are at least enough bytes for a header
if (serialPort.BytesToRead >= 6) {
byte[] databytes = new byte[6];
serialPort.Read(databytes, 0, 6);
// Parse the header - you have to create this function
int calculatedMsgSize = ValidateHeader(databytes);
byte [] msg = new byte[calculatedMsgSize];
int bytesRead = 0;
while (bytesRead < calculatedMsgSize) {
if (serialPort.BytesToRead) {
bytesRead += serialPort.Read(msg, bytesRead,
Math.min(calculatedMsgSize - bytesRead, serialPort.BytesToRead));
}
}
// You should now have a complete message
HandleMsg(msg);
}
I've an UART device which I'm writing to it a command (via System.IO.Ports.SerialPort) and then immediately the device will respond.
So basically my approach is:
->Write to SerialPort->await Task.Delay->Read from the Port.
//The port is open all the time.
public async byte[] WriteAndRead(byte[] message){
port.Write(command, 0, command.Length);
await Task.Delay(timeout);
var msglen = port.BytesToRead;
if (msglen > 0)
{
byte[] message = new byte[msglen];
int readbytes = 0;
while (port.Read(message, readbytes, msglen - readbytes) <= 0)
;
return message;
}
This works fine on my computer. But if I try it on another computer for example, the bytesToRead property is sometimes mismatched. There are empty bytes in it or the answer is not completed. (E.g. I get two bytes, if I expect one byte: 0xBB, 0x00 or 0x00, 0xBB)
I've also looked into the SerialPort.DataReceived Event, but it fires too often and is (as far as I understand) not really useful for this write and read approach. (As I expect the answer immediately from the device).
Is there a better approach to a write-and-read?
Read carefully the Remarks in https://msdn.microsoft.com/en-us/library/ms143549(v=vs.110).aspx
You should not rely on the BytesToRead value to indicate message length.
You should know, how much data you expect to read to decompose the message.
Also, as #itsme85 noticed, you are not updating the readbytes, and therefore you are always writing received bytes to beginning of your array. Proper code with updating the readbytes should look like this:
int r;
while ((r = port.Read(message, readbytes, msglen - readbytes)) <= 0){
readbytes += r;
}
However, during the time you will read data, more data can come and your "message" might be incomplete.
Rethink, what you want to achieve.
On receiving Multipart data from the browser (which has a size of greater than ~2KB), I start receiving null '\0' bytes after the first few chunks which are relevant when I use:
_Stream.Read(ByteArray, Offset, ContentLength);
But, if I divide the ContentLength into small buffers (around 2KB each) AND add a delay of 1ms after each call to Read(), then it works fine:
for(int i = 0; i < x; i++)
{
_Stream.Read(ByteArray, Offset * i, BufferSize);
System.Threading.Thread.Sleep(1);
}
But, adding delay is quite slow. How to prevent reading null bytes. How can I know how many bytes have been written by the browser.
Thanks
The 0x00 bytes were not actually received, they were never written to.
Stream.Read() returns the number of bytes actually read, which is in your case often less than BufferSize. Small amounts of data typically arrive in a single message, in which case the problem does not occur.
The delay might "work" in your test scenario because by then the network layer has buffered more than BufferSize data. It will probably fail in a production environment.
So you'll need to change your code into something like:
int remaining = ContentLength;
int offset = 0;
while (remaining > 0)
{
int bytes = _Stream.Read(ByteArray, offset, remaining);
if (bytes == 0)
{
throw new ApplicationException("Server disconnected before the expected amount of data was received");
}
offset += bytes;
remaining -= bytes;
}
I am trying to send various bits of PC information such as free HDD space, total RAM etc to a Windows Service over TCP. I have the following code which basically creates a string of information split by a |, ready for processing within the Windows Service TCP server to be put in to a SQL table.
Is it best to do this as I have done or is there a better way?
public static void Main(string[] args)
{
Program stc = new Program(clientType.TCP);
stc.tcpClient(serverAddress, Environment.MachineName.ToString() + "|" + FormatBytes(GetTotalFreeSpace("C:\\")).ToString());
Console.WriteLine("The TCP server is disconnected.");
}
public void tcpClient(String serverName, String whatEver)
{
try
{
//Create an instance of TcpClient.
TcpClient tcpClient = new TcpClient(serverName, tcpPort);
//Create a NetworkStream for this tcpClient instance.
//This is only required for TCP stream.
NetworkStream tcpStream = tcpClient.GetStream();
if (tcpStream.CanWrite)
{
Byte[] inputToBeSent = System.Text.Encoding.ASCII.GetBytes(whatEver.ToCharArray());
tcpStream.Write(inputToBeSent, 0, inputToBeSent.Length);
tcpStream.Flush();
}
while (tcpStream.CanRead && !DONE)
{
//We need the DONE condition here because there is possibility that
//the stream is ready to be read while there is nothing to be read.
if (tcpStream.DataAvailable)
{
Byte[] received = new Byte[512];
int nBytesReceived = tcpStream.Read(received, 0, received.Length);
String dataReceived = System.Text.Encoding.ASCII.GetString(received);
Console.WriteLine(dataReceived);
DONE = true;
}
}
}
catch (Exception e)
{
Console.WriteLine("An Exception has occurred.");
Console.WriteLine(e.ToString());
}
}
Thanks
Because TCP is stream-based, it is important to have some indicator in the message to signal the other end when it has read the complete message. There are two traditional ways of doing this. First, you could have some special byte pattern at the end of each message. When the other end reads the data, it knows that it has read a full message when that special byte pattern is seen. Using this mechanism requires a byte pattern that is not likely to be included in the actual message. The other way is to include the length of the data at the beginning of the message. This is the way I do it. All my TCP messages include a short header structured like this:
class MsgHeader
{
short syncPattern; // e.g., 0xFDFD
short msgType; // useful if you have different messages
int msgLength; // length of the message minus header
}
When the other side starts receiving data, it reads the first 8 bytes, verifies the sync pattern (for the sake of sanity), and then uses the message length to read the actual message. Once the message has been read, it processes the message based on the message type.
I'd suggest creating a class that gathers the system information you're interested in and is capable of encoding/decoding it, something like:
using System;
using System.Text;
class SystemInfo
{
private string machineName;
private int freeSpace;
private int processorCount;
// Private so no one can create it directly.
private SystemInfo()
{
}
// This is a static method now. Call SystemInfo.Encode() to use it.
public static byte[] Encode()
{
// Convert the machine name to an ASCII-based byte array.
var machineNameAsByteArray = Encoding.ASCII.GetBytes(Environment.MachineName);
// *THIS IS IMPORTANT* The easiest way to encode a string value so that it
// can be easily decoded is to prepend the length of the string. Otherwise,
// you're left guessing on the decode side about how long the string is.
// Calculate the message length. This does *NOT* include the size of
// the message length itself.
// NOTE: As new fields are added to the message, account for their
// respective size here and encode them below.
var messageLength = sizeof(int) + // length of machine name string
machineNameAsByteArray.Length + // the machine name value
sizeof(int) + // free space
sizeof(int); // processor count
// Calculate the required size of the byte array. This *DOES* include
// the size of the message length.
var byteArraySize = messageLength + // message itself
sizeof(int); // 4-byte message length field
// Allocate the byte array.
var bytes = new byte[byteArraySize];
// The offset is used to keep track of where the next field should be
// placed in the byte array.
var offset = 0;
// Encode the message length (a very simple header).
Buffer.BlockCopy(BitConverter.GetBytes(messageLength), 0, bytes, offset, sizeof(int));
// Increment offset by the number of bytes added to the byte array.
// Note that the increment is equal to the value of the last parameter
// in the preceding BlockCopy call.
offset += sizeof(int);
// Encode the length of machine name to make it easier to decode.
Buffer.BlockCopy(BitConverter.GetBytes(machineNameAsByteArray.Length), 0, bytes, offset, sizeof(int));
// Increment the offset by the number of bytes added.
offset += sizeof(int);
// Encode the machine name as an ASCII-based byte array.
Buffer.BlockCopy(machineNameAsByteArray, 0, bytes, offset, machineNameAsByteArray.Length);
// Increment the offset. See the pattern?
offset += machineNameAsByteArray.Length;
// Encode the free space.
Buffer.BlockCopy(BitConverter.GetBytes(GetTotalFreeSpace("C:\\")), 0, bytes, offset, sizeof(int));
// Increment the offset.
offset += sizeof(int);
// Encode the processor count.
Buffer.BlockCopy(BitConverter.GetBytes(Environment.ProcessorCount), 0, bytes, offset, sizeof(int));
// No reason to do this, but it completes the pattern.
offset += sizeof(int).
return bytes;
}
// Static method. Call is as SystemInfo.Decode(myReceivedByteArray);
public static SystemInfo Decode(byte[] message)
{
// When decoding, the presumption is that your socket code read the first
// four bytes from the socket to determine the length of the message. It
// then allocated a byte array of that size and read the message into that
// byte array. So the byte array passed into this function does *NOT* have
// the 4-byte message length field at the front of it. It makes no sense
// in this class anyway.
// Create the SystemInfo object to be populated and returned.
var si = new SystemInfo();
// Use the offset to navigate through the byte array.
var offset = 0;
// Extract the length of the machine name string since that is the first
// field encoded in the message.
var machineNameLength = BitConverter.ToInt32(message, offset);
// Increment the offset.
offset += sizeof(int);
// Extract the machine name now that we know its length.
si.machineName = Encoding.ASCII.GetString(message, offset, machineNameLength);
// Increment the offset.
offset += machineNameLength;
// Extract the free space.
si.freeSpace = BitConverter.ToInt32(message, offset);
// Increment the offset.
offset += sizeof(int);
// Extract the processor count.
si.processorCount = BitConverter.ToInt32(message, offset);
// No reason to do this, but it completes the pattern.
offset += sizeof(int);
return si;
}
}
To encode the data, call the Encode method like this:
byte[] msg = SystemInfo.Encode();
To decode the data once it's been read from the socket, call the Decode method like this:
SystemInfo si = SystemInfo.Decode(msg);
As to your actual code, I'm not sure why you're reading from the socket after writing to it unless you're expecting a return value.
A few things to consider. Hope this helps.
EDIT
First of all, use the MsgHeader if you feel you need it. The example above simply uses the message length as the header, i.e., it does not include a sync pattern or a message type. Whether you need to use this additional information is up to you.
For every new field you add to the SystemInfo class, the overall size of the message will increased, obviously. Thus, the messageLength value needs to be adjusted accordingly. For example, if you add an int to include the number of processors, messageLength will increase by sizeof(int). Then, to add it to the byte array, simply use the same System.Buffer.BlockCopy call. I've adjusted the example to show this with a little more detail, including making the method static.
I tried to understand the MSDN example for NetworkStream.EndRead(). There are some parts that i do not understand.
So here is the example (copied from MSDN):
// Example of EndRead, DataAvailable and BeginRead.
public static void myReadCallBack(IAsyncResult ar ){
NetworkStream myNetworkStream = (NetworkStream)ar.AsyncState;
byte[] myReadBuffer = new byte[1024];
String myCompleteMessage = "";
int numberOfBytesRead;
numberOfBytesRead = myNetworkStream.EndRead(ar);
myCompleteMessage =
String.Concat(myCompleteMessage, Encoding.ASCII.GetString(myReadBuffer, 0, numberOfBytesRead));
// message received may be larger than buffer size so loop through until you have it all.
while(myNetworkStream.DataAvailable){
myNetworkStream.BeginRead(myReadBuffer, 0, myReadBuffer.Length,
new AsyncCallback(NetworkStream_ASync_Send_Receive.myReadCallBack),
myNetworkStream);
}
// Print out the received message to the console.
Console.WriteLine("You received the following message : " +
myCompleteMessage);
}
It uses BeginRead() and EndRead() to read asynchronously from the network stream.
The whole thing is invoked by calling
myNetworkStream.BeginRead(someBuffer, 0, someBuffer.Length, new AsyncCallback(NetworkStream_ASync_Send_Receive.myReadCallBack), myNetworkStream);
somewhere else (not displayed in the example).
What I think it should do is print the whole message received from the NetworkStream in a single WriteLine (the one at the end of the example). Notice that the string is called myCompleteMessage.
Now when I look at the implementation some problems arise for my understanding.
First of all: The example allocates a new method-local buffer myReadBuffer. Then EndStream() is called which writes the received message into the buffer that BeginRead() was supplied. This is NOT the myReadBuffer that was just allocated. How should the network stream know of it? So in the next line numberOfBytesRead-bytes from the empty buffer are appended to myCompleteMessage. Which has the current value "". In the last line this message consisting of a lot of '\0's is printed with Console.WriteLine.
This doesn't make any sense to me.
The second thing I do not understand is the while-loop.
BeginRead is an asynchronous call. So no data is immediately read. So as I understand it, the while loop should run quite a while until some asynchronous call is actually executed and reads from the stream so that there is no data available any more. The documentation doesn't say that BeginRead immediately marks some part of the available data as being read, so I do not expect it to do so.
This example does not improve my understanding of those methods. Is this example wrong or is my understanding wrong (I expect the latter)? How does this example work?
I think the while loop around the BeginRead shouldn't be there. You don't want to execute the BeginRead more than ones before the EndRead is done. Also the buffer needs to be specified outside the BeginRead, because you may use more than one reads per packet/buffer.
There are some things you need to think about, like how long are my messages/blocks (fixed size). Shall I prefix it with a length. (variable size) <datalength><data><datalength><data>
Don't forget it is a Streaming connection, so multiple/partial messages/packets can be read in one read.
Pseudo example:
int bytesNeeded;
int bytesRead;
public void Start()
{
bytesNeeded = 40; // u need to know how much bytes you're needing
bytesRead = 0;
BeginReading();
}
public void BeginReading()
{
myNetworkStream.BeginRead(
someBuffer, bytesRead, bytesNeeded - bytesRead,
new AsyncCallback(EndReading),
myNetworkStream);
}
public void EndReading(IAsyncResult ar)
{
numberOfBytesRead = myNetworkStream.EndRead(ar);
if(numberOfBytesRead == 0)
{
// disconnected
return;
}
bytesRead += numberOfBytesRead;
if(bytesRead == bytesNeeded)
{
// Handle buffer
Start();
}
else
BeginReading();
}