I am writing a simple app where an Ardunio sends read data to the C# program using simple serial.write()'s. I send the data from Ardunio with special syntax and the c# program reads and interprets it and eventually graphs the data.
For example, the Ardunio program sends data
Serial.write("DATA 1 3 90;");
which means: X value and Y value in chart 1 is 3 and 90 respectively. And the c# program reads it using
private async void Serialport_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
await serialport.BaseStream.ReadAsync(buffer, 0, 1024);
await Task.Run(() => Graph()); // Graph function graphs the data
}
In the graph() function, I convert the buffer to a string with Encode.ASCII.GetString(buffer); and interpret the data with the syntax and stuff. But for some reason either the c# program doesn't read it fast enough or the Arduino doesn't send it fast enough and the messages are sometimes interrupted For example the packet I received is:
DATA 2 5 90;DATA 5 90 10;DATA 1 1 <|-------- I cannot get the last Y Value
And the next data module starts with
75;DATA 5 4 60;DATA 14 5 6;DATA
/\
|
+==================== It is here
BTW, all the packets are 32 bytes.
So I either need to get the data line by line but I cannot do that because ardunio sends it too fast
Serial.write("DATA 1 3 90;");
Serial.write("DATA 2 4 40;");
Comes to C# as DATA 1 3 90;DATA 2 4 40; as a whole block. Or I need to get it all at once?
(I prefer getting it line by line)
UPDATE:
When delay(1000); is added between sends. The data is processed correctly. Without the delays, Arduino sends the data too fast, and data clumps and interrupted. How can I make sure that there is no delay in the data yet the data is reliable and without interruption?
UPDATE 2:
When buffer size is increased to 100 * 1024 * 1024 as well as the readCount in ReadAsync method, the read message is much larger still with interruptions tho.
I can give you any extra information.
UPDATE 3:
I ll try to answer the comments;
First of all, I am already doing that and I forgot to mention it. I split the incoming data from the semicolons and interpret them one by one with a foreach loop
Second I am using buffer globally instead of passing it as an argument to graph function
Third I ll try Serial.PrintLine and give you feedback on it
UPDATE 4:
I was able to solve the problem by replacing ReadAsync with Serial.ReadLine() and serial.write with serial.printline(); The solution is suggested by Hans Passant
UPDATE 5:
I still encounter the same problem but differently. The graph function is async so when I write the data so fast I miss some data trying to graph and read other data.
Also because the graph function is async chart control plots the points without order and the graph gets messy.
This can be easily solvable if I add delay(50) between Serial.println()'s
PS. I didn't give the whole code because it is a large block. But I can give it piece by piece if you tell me where you want it.
Any help is appreciated.
Related
I am writing a simple app where an Ardunio sends read data to the C# program using simple serial.write()'s. I send the data from Ardunio with special syntax and the c# program reads and interprets it and eventually graphs the data.
For example, the Ardunio program sends data
Serial.write("DATA 1 3 90;");
which means: X value and Y value in chart 1 is 3 and 90 respectively.
And the c# program reads it using
private async void Serialport_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
await serialport.BaseStream.ReadAsync(buffer, 0, 1024);
await Task.Run(() => Graph()); // Graph function graphs the data
}
In the graph() function, I convert the buffer to a string with Encode.ASCII.GetString(buffer); and interpret the data with the syntax and stuff. But for some reason either the c# program doesn't read it fast enough or the Arduino doesn't send it fast enough and the messages are sometimes interrupted For example the packet I received is:
DATA 2 5 90;DATA 5 90 10;DATA 1 1 <|-------- I cannot get the last Y Value
And the next data module starts with
75;DATA 5 4 60;DATA 14 5 6;DATA
/\
|
+==================== It is here
BTW, all the packets are 32 bytes.
So I either need to get the data line by line but I cannot do that because ardunio sends it too fast
Serial.write("DATA 1 3 90;");
Serial.write("DATA 2 4 40;");
Comes to C# as DATA 1 3 90;DATA 2 4 40; as a whole block.
Or I need to get it all at once?
(I prefer getting it line by line)
UPDATE:
When delay(1000); is added between sends. The data is processed correctly. Without the delays, Arduino sends the data too fast, and data clumps and interrupted. How can I make sure that there is no delay in the data yet the data is reliable and without interruption?
UPDATE 2:
When buffer size is increased to 100 * 1024 * 1024 as well as the readCount in ReadAsync method, the read message is much larger still with interruptions tho.
I can give you any extra information.
PS. I didn't give the whole code because it is a large block. But I can give it piece by piece if you tell me where you want it.
Any help is appreciated.
I have a one-on-one connection between a server and a client. The server is streaming real-time audio/video data.
My question may sound weird, but should I use multiple ports/socket or only one? Is it faster to use multiple ports or a single one offer better performance? Should I have a port only for messages, one for video and one for audio or is it more simple to package the whole thing in a single port?
One of my current problem is that I need to first send the size of the current frame as the size - in bytes - may change from one frame to the next. I'm fairly new to Networking, but I haven't found any mechanism that would automatically detect the correct range for a specific object being transmitted. For example, if I send a 2934 bytes long packet, do I really need to tell the receiver the size of that packet?
I first tried to package the frame as fast as they were coming in, but I found out the receiving end would sometime not get the appropriated number of bytes. Most of the time, it would read faster than I send them, getting only a partial frame. What's the best way to get only the appropriated number of bytes as quickly as possible?
Or am I looking too low and there's a higher-level class/framework used to handle object transmission?
I think it is better to use an object mechanism and send data in an interleaved fashion. This mechanism may work faster than multiple port mechanism.
eg:
class Data {
DataType, - (Adio/Video)
Size, - (Size of the Data buffer)
Data Buffer - (Data depends on the type)
}
'DataType' and 'Size' always of constant size. At the client side take the 'DataType' and 'Size' and then read the specifed size of corresponding sent data(Adio/Video).
Just making something up off the top of my head. Shove "packets" like this down the wire:
1 byte - packet type (audio or video)
2 bytes - data length
(whatever else you need)
|
| (raw data)
|
So whenever you get one of these packets on the other end, you know exactly how much data to read, and where the beginning of the next packet should start.
[430 byte audio L packet]
[430 byte audio R packet]
[1000 byte video packet]
[20 byte control packet]
[2000 byte video packet]
...
But why re-invent the wheel? There are protocols to do these things already.
I'm one of those guys who come here to find answers to those questions that others have asked, and I think i newer asked anything myself, but after two days searching unsuccessfully I decided that it's time to ask something myself. So here it is...
I have a TCP server and client written in C#, .NET 4, asynchronous sockets using SocketAsyncEventArgs. I have a length-prefixed message framing protocol. Overall everything works just fine, but one issue keeps bugging me.
Situation is like this (I will use small numbers just as an example):
Lets say Server has a Send buffer length of 16 bytes.
It sends a message which is 6 bytes long, and prefixes it with 4 bytes long length prefix. Total message length is 6+4=10.
Client reads the data and receives a buffer of 16 bytes length (yes 10 bytes of data and 6 bytes equal to zero).
Received buffer looks like this: 6 0 0 0 56 21 33 1 5 7 0 0 0 0 0 0
So I read first 4 bytes which is my length prefix, I determine that my message is 6 bytes long, I read it as well and everything is fine so far. Then i have 16-10=6 bytes left to read. All of them are zeroes I read 4 of them, since it's my length prefix. So it's a zero length message which is allowed as keep-alive packet.
Remaining data to read: 0 0
Now the issue "kicks in". I got only 2 remaining bytes to read, they are not enough to complete a 4 byte-long length prefix buffer. So I read those 2 bytes, and wait for more incoming data. Now server is not aware that I'm still reading length prefix (I'm just reading all those zeroes in the buffer) and sends another message correctly prefixed with 4 bytes. And the client is assuming the server sends those missing 2 bytes. I receive the data on the client side, and read first two bytes to form a complete 4 byte length buffer. The results are something like that
lengthBuffer = new byte[4]{0, 0, 42, 0}
Which then translates into 2752512 message length. So my code will continue to read next 2752512 bytes to complete the message...
So in every single message framing example I have seen zero length messages are supported as keep-alive's. And every example I've seen doesn't do anything more than I do. The problem is that I do not know how much data I have to read when I receive it from the server. Since I have partially-filled buffer with zeroes, I have to read it all as those zeroes could be keep-alive's I sent from the other end of connection.
I could drop zero-length messages and stop reading the buffer after first empty message and it should fix this issue, and use custom messages for my keep-alive mechanism. But I want to know if I am missing something, or doing something wrong, since every code example I've seen seems to have same issue (?)
UPDATE
Marc Gravell, you sir pulled words out of my mouth. Was about to update that the issue is with sending the data. The problem is that initially when exploring .NET Sockets and SocketAsyncEventArgs I came across this sample: http://archive.msdn.microsoft.com/nclsamples/Wiki/View.aspx?title=socket%20performance
It uses reusable pool of buffers. Simply takes predefined number of maximum client connections allowed, for example 10, takes maximum single buffer size, for example 512, and creates one large buffer for all of them. So 512 * 10 * 2 (for send and receive) = 10240
So we have byte[] buff = new byte[10240];
Then for each client that connects it assigns a piece of this large buffer. First connected client gets first 512 bytes for Data Reading operations, and gets next 512 bytes (offset 512) for Data Sending operations. Therefore the code ended up having already allocated Send buffer which size is 512 (exactly the number the client later receives as BytesTransferred). This buffer is populated with data, and all remaining space out of these 512 bytes is sent as zeroes.
Strange enough this example is from msdn. The reason there is a single huge buffer is to avoid fragmented heap memory, when buffer gets pinned and GC cant collect it or something like that.
Comment from BufferManager.cs in the provided example (see link above):
This class creates a single large buffer which can be divided up and
assigned to SocketAsyncEventArgs objects for use with each socket I/O
operation. This enables bufffers to be easily reused and gaurds
against fragmenting heap memory.
So the issue is pretty much clear. Any suggestions on how I should resolve this are welcome :) Is it true what they say about fragmented heap memory, is it OK to create a data buffer "on the fly"? If so, will I have memory issues when the server scales to a few hundred or even thousands of clients?
I guess the problem is that you are treating the trailing zeros in the buffer you read as data. This is not data. It is garbage. No one ever sent it to you.
The Stream.Read call returns you the number of bytes actually read. You should not interpret the rest of the buffer in any way.
The problem is that I do not know how much data I have to read when I
receive it from the server.
Yes, you do: Use the return value from Stream.Read.
That sounds simply like a bug in either your send or receive code. You should only get BytesTransferred as the data that was actually sent, or some number smaller than that if arriving in fragments. The first thing I would wonder is: did you setup the send correctly? i.e. if you have an oversized buffer, a correct implementation might look like:
args.SetBuffer(buffer, 0, actualBytesToSend);
if (!socket.SendAsync(args)) { /* whatever */ }
where actualBytesToSend can be much less than buffer.Length. My initial suspicion is that
you are doing something like:
args.SetBuffer(buffer, 0, buffer.Length);
and therefore sending more data than you have actually populated.
I should emphasize: there is something wrong in either your send or receive; I do not believe, at least without an example, that there is some fundamental underlying bug in the BCL here - I use the async API extensively, and it works fine - but you do need to accurately track the data you are sending and receiving at all points.
"Now server is not aware that I'm still reading length prefix (I'm just reading all those zeroes in the buffer) and sends another message correctly prefixed with 4 bytes.".
Why? How does the server know what you are and aren't reading? If the server retransmits any part of a message it is in error. TCP already does that for you.
There seems to be something radically wrong with your server.
If I have a port that I need to read from quite a lot, and the returning amount of data varies from one line to many lines, what is the best way to read / sift through what I am reading?
After some research I found out what I was supposed to do.
string Xposition;
Xposition = "";
threeAxisPort.ReadExisting();
threeAxisPort.WriteLine("rx");
Thread.Sleep(50);
Xposition = threeAxisPort.ReadExisting();
threeAxisPort.WriteLine("end");
What I ended up doing was clearing the port of everything usign a ReadExisting function, then waited 50 ms to not flood the port, and did another ReadExisting command. With the additional 50ms wait time there was no leftover from the motors, and the returning block of text was exactly what I needed, however, I will still be thinking up of a more dynamic way to handle that string, because in a worst case scenario the read existing will pick up something I won't want.
I have a serial device that has a binary output and I capture the data using the following.
private void port_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
int count = sp.BytesToRead;
byte[] data = new byte[count];
sp.Read(data, 0, data.Length);
file.WriteLine(BitConverter.ToString(data));
}
The data comes through and looks like this...
06-14-F2-A1-64-2D-62-00-1A-31-00-06-14-F3-84-62-59-01-00-1A-31-00-06-14-F3-85-56-52-55-31
1A-31-00-06-14-F4-18-04-2E-62-00-1A-31-00-06-14-F4-E3-27-5B-01-00-1A-31-00-06-14-F4-E4-1C-51-55-31
1A-31-00-06-14-F5-71-4C-59-71-20-1A-31-00-06-14-F5-8E-A5-2E-62-00-1A-31-00-06-14-F5-F4-47-56-55-31-1A-31-00-06-14-F6-10-1A-1A-31-52-24-1A-31-00-06-14-F6-3D-40-19-70-00-1A-31-00-06-14-F6-3E-9C-4C-55-31-1A-33-00-06-14-F6-F6-11-3D-A0-00-17-B0-C8-4E-42-70-AA-00-00-59-51-1E-1A-31-00-06-14-F7-05-4A-2E-62-00-1A-31-00-06-14-F7-83-5C-56-55-31-1A-31-00-06-14-F7-99-04-5A-01-00-1A-31-00-06-14-F7-99-F8-51-55-31-1A-31-00-06-14-F8-7B-EA-2E-62-00-1A-31-00-06-14-F9-00-CE-56-01-00-1A-31-00-06-14-F9-0E-DF-51-55-31-1A-31-00-06-14-F9-F2-8B-2B-62-00-1A-31-00-06-14-FA-15-1F-1D-05-30-1A-31-00-06-14-FA-62-4D-59-01-00-1A-31-00-06-14-FA-63-41-55-55-31-1A-31-00-06-14-FA-6F-6E-1D-67-67-1A-31-00-06-14-FA-EC-50-2E-72-00-1A-31-00-06-14-FB-22-96-38-62-00-1A-31-00-06-14-FB-3B-7A-40-20-43-1A-31-00-06-14-FB-69-2E-2B-62-00-1A-31-00-06-14-FC-62-F1-2D-72-00-1A-31-00-06-14-FC-DF-D1-2E-62-00-1A-31-00-06
The hex isn't the issue here as I can decode that but a statement I am looking for begins with 1A-31 and then is a set number of bytes long. As you can see the serial stream in this case starts mid flow and so is not a full statement.
How can I look for this marker, discard the beginning and then start processing. Also bear in mind that this will happen multiple times as the readBuffer at some point will truncate the stream and I will need to piece it back together again?
You're almost there. Your problem is that the data you're streaming comes in chunks, which misalign with where statements begin and end. I'm going to assume that the end of a statement is found by the 1A-31 that identifies the start of the next statement. If this isn't true, reinterpret this answer accordingly.
Now, you will not be able to do anything with the very first pieces of data in your example, which contains half a statement. So, let's start with assuming that the first chunk of data you get indeed starts with 1A-31.
There are now two options:
You can find the entire statement inside the chunk (i.e. you encounter another 1A-31 inside it). In this case, eat it up and do with it whatever you'd want to do with it (I'd add a StatementReceived event and send it there, or something like that). Repeat this exercise until the chunk has been entirely processed.
The statement is not entirely contained inside the chunk. Copy the data you already got to a temporary buffer and wait for the next port_DataReceived call.
If the second option was the case, you know that the data for the next port_DataReceived will not begin with 1A-31 (because the temporary buffer is non-empty). However, you can scan to the end of it (until the next 1A-31), prepend the temporary buffer (stored in the previous port_DataReceived call) to it, and raise StatementReceived and erase the temporary buffer.
With a similar approach, you can also deal with statements that require more than 2 chunks of data to be sent; each time you do not encounter a 1A-31, append the received data to the temporary buffer, until the statement is complete.
Finally, if the very first bytes that you read upon startup do not start with 1A-31, you'll just have to discard those. Can't do something with half a statement.