I'm currently working on an application that communicates with an electronic device via the SerialPort.
This communication is done in a half-duplex fashion, where the application is the master and the device is the slave. The master needs to send a message to the device, and the device needs to respond before the next message is sent.
If the message doesn't receive a response, the application needs to resend it.
The content of the next message is dependent on the result of the current message. i.e. each new message has an incremented sequence number, and sometimes data for the next message is taken from the reponse of the current one.
To send messages I use an interface to System.IO.Ports.SerialPort. When messages are received a SerialDataReceivedEventHandler is fired.
What's the best way for me to manage this? Is there a pattern that I can base this on?
I have worked on something similar in the past. This is the basic structure of the messaging I have used:
Application: Send Command Seq #1 -->
<-- Device: Acknowledge command, seq #1 (including any command-specific response)
If the device didn't acknowledge within 1 second, the same command would be resent, same sequence number. This "retry" sequence would happen 3 times and then it would time out, and retire the command.
The sequence number from the application side would increment by one for every command it sent that was not a retry. It would loop back to sequence number 1 after hitting 99.
The application will have to keep track of its state based on what command(s) is "in-flight" between it and the device, and what kind of response it has received. It can identify the response to a specific command by the sequence number the device puts in its acknowledgement.
To keep it simple to transition from state to state you can make it so that there is only ever one active command and don't move on until that one has been ack'ed or timed out and retired.
Shane Wealti approach is right but I think what you might be really asking yourself, should you use threads? Even though it is a master/slave scenario, should you have a listening thread? When do you listen to the port and when do you send?
The simplest approach - no threads
You don't need to use threads in this scenario due to master/slave configuration, all you have is two functions.
SendCommand(char * bfr)
{
}
RecieveCommand(char * bfr)
{
}
SendCommand( txBfr );
RecieveData( rxBfr );
// process receive buffer, prepare new command
SendCommand( txbfr );
RecieveCommand( rxBfr );
// and so on
The approach is the most simple one and totally functional. However since there are no threads and say your RecieveData() times out in one second, you GUI will not be responsive in that second. Note that you are not listening to the port all the time but only when are expecting a reply.
I might edit this to add Comprehensive approach using threads later but don't have the time right now.
Related
SenderApp is sending values to service bus queue and on the other end we have two receiver instances.
Requirement is that we only save values that have changed to db (all incoming values are first saved to redis cache where comparison happens).
SenderApp sends three values in following order to queue: (1, 2, 1).
-----1---2---1------------------------>
Now the values go in queue with FIFO method and on the other end of queue we have two instances of receiver application.
This is where it gets interesting.
Lets say due to latency or some other factor second receiver instance is slow to process the value(2) and ends up saving it to database last from all three values.
So it should be something like this:
Receiver instance #1
---------------------1---1------------>
Receiver instance #2
-----------------------------2-------->
Now we have a problem. Instance one is comparing the second sent value which is 1 against the first value which is also 1 and it doesn't get saved to database. Values that are sent to service bus queue have timestamps attached to them.
Also it needs to be fairly scalable solution.
All ideas are welcome, maybe levarage the redis cache, maybe service bus sessions?
Edit:
For more clarification:
Each incoming message has device id and value attached to it so we must not save any consecutive duplicate values for that specific device.
So for example we have two incoming messages from the same device.
First incoming message has a value 1 with device id 999 (we must save it).
But now the next incoming message also has value 1 and device id 999.
So that would be consecutive duplicate and we must not save it.
Also what makes this problem difficult is that we also can not save values directly on sender side.
Explanatory graph of the general flow below:
Competing consumers (receivers) will contradict the requirement to handle messages in the order they were sent. To adhere to the in-order processing, the Azure Service Bus message sessions feature could help. A single receiver only processes a session at any time; there's no processing by multiple receivers. This eliminates the chance of having messages from the same source processed in parallel and out of order. This is still a scalable approach as receivers can handle different sessions. If messages for a given session arrive after some time, the session processing would be picked up by any competing receivers.
In this scenario, where a uniquely identifiable device emits messages, the device ID could be used as a session ID.
Worth noting that each receiver can handle multiple sessions in parallel, not just a single session. That way, the solution can scale out and up, depending on the compute used and complexity of the code executed per message.
We use rabbit mq to send messages to a server for processing.
We require the server to ack a message. That way if the server happens to die whilst processing the message, we will retry the message when it restarts, or with a different server.
The problem is, on a very rare occasion, we will get a message that deterministically crashes the server. This is because we call into some open source native dlls, those dlls have bugs, and sometimes these dlls just cause the process to crash with no exception. Of course it would be ideal to fix those bugs, but we don't expect to fix all such issues in pdfium or opencv any time soon. We have to reckon with the fact that whatever we do, we will eventually get such a message.
The result of this is that the message is then retried, the server restarts, picks ups the message, crashes, and so on ad infinitum. Nothing gets processed till we manually stop the server, and purge the message. Not ideal.
What can we do to solve this problem?
What we don't want to do is create another service that monitors the rabbitmq service, looks for such messages and purges them, since that just leads to spiralling complexity. Instead we want to deal with this at the rabbitmq client level. We would be perfectly happy to say that if a message is not processed 3 times, we should just fail the message. We could do this by maintaining a database entry of which messages we've processed, but ideally I wouldn't want to involve anything external, and just contain the solution to this problem in our rabbitmq client library. I'm not sure how to do this though.
One method I have used in my event driven architecture is to use dead letter exchanges (DLXs) or poison queues, that way if we see the same message multiple times due to service failure then it'll be pushed into the DLX instead of being re-queued into the original exchange. These messages then trigger a different type of process within our system to alert us messages are stuck and failing to process, we can then diagnose and fix the consumer. After a fix has been made we trigger another process to move the poison messages back into the original exchange to be then processed as normal.
In your scenario because your process crashes there is two possible options to deal with these messages:
If the message is marked as redelivered then clone the message and add an attempt count to the body or as a header (x-attempt-count) to the message. The copy will then be added to the back of the queue with the attempt count. When the copy is then consumed you can check if it hits the threshold and then move the message into a DLX or store in a database. The major drawback here is that it breaks the order of which the messages are processed.
Use an external services to keep track of the number of delivery attempts, I would recommend using something like redis/memcache where you can increment a counter based on a unique message id. At the start of your process if the message has been marked as redelivered then lookup the counter. If the message has reached the threshold, trigger a different process again like moving it into a DLX.
I am seeking advice on a project I have been assigned and I'm looking to see how it's done "professionally," or any suggestions that can lead me to the right direction.
I have a server piece that accepts commands from clients and pushes out byte streams to a serial port. Although multiple clients can send commands to this server piece, our hardware only can handle one command at a time. My problem is with queueing in the software end.
I've implemented a Queue<T> helper class that also inserts data into a DataSet containing: the requesting client number, message data (byte array to write to serial port) and message type (command description). It will also list the queue commands in a DataGrid (on the Form). Probably not the way to go, but that's the only thing I can think of as far as retaining the requesting client and the data and showing, visually, the queue.
Where do I handle the processing of the queue? I thought about handling it on a custom event where if the DataGrid list changed (item added/removed), grab the first row of data in the DataSet and send it out to the serial port.
Any comments or suggestions are greatly appreciated.
Thanks.
Edit: I forgot to add that it does require a response from the SerialPort as well, in order for the current executed command to be removed from the queue.
I would use a database table to store the queue of commands. The web app would add records to the queue and display the queue, then a separate process (such as a Windows service or console app) would request the next command from the database and send it to the serial port.
Client requests can come in at any time, they'll probably be handled by some proxy class (WCF?) on its own thread/task. Then that thread/ task needs to coordinate with the task that's 'inside' the model actually processing the requests.
A good class to do this with is the BlockingCollection.
The server-thread will block until there's something in the collection to work on. It can then take it from the collection in a thread safe manner and process it. Doing it this way ensures that the requests can be accepted when they arrive, but they are processed on at a time.
The overall pattern to think of here is producer-consumer.
GJ
If it is a high trasaction web application you might want to look at queueing system such as MSMQ, Service Broker Queue or RabbitMQ. Window service can then pick up the queued items and send it to serial port.
I can use some help for a designing my COMport connection classes.
I have a device with a microcontroller (which I programmed) connected to my comport.
Whenever I send a message to it, it should send an acknowledge.
So whenever I send something over the comport, it should wait for an acknowledge before continuing.
Ofcourse, I dont want something like a while(!ack) wait().
When a ack is not received in 1 second or so, it should send the command again.
The Ack looks different for each type of command (note: only the type, no message-specific id).
The connected device also sends messages (apart from ACKs), which need to be handled by the application itself.
Does anybody has suggestions about an easy and flexible way (a design pattern maybe? a sample?) to fix this?
You'll probably want a dedicated thread that handles the communications. You'll need a Queue on which the client code can push a message, protect it with a ReaderWriterLockSlim. No need for the DataReceived event, just call SerialPort.Read() directly. Detect timeouts with the ReadTimeout property. If you get responses that need to go back to the client code then use an event.
Watch out designing the protocol, it isn't that easy to get right. You'll protect against loss of bytes with your scheme, but it is just as likely for the ACK to be lost. The microcontroller will see the same command twice. Now you need a "message number" to suppress duplicates and a way for both ends to synchronize them. Take a look at RFC 916.
I'm an embedded programmer trying to do a little bit of coding for a communications app and need a quick start guide on the best / easiest way to do something.
I'm successfully sending serial data packets but need to impliment some form of send/ response protocol to avoid overflow on the target system and to ensure that the packet was received ok.
Right now - I have all the transmit code under a button click and it sends the whole lot without any control.
What's the best way to structure this code , i.e sending some packets - waiting for response .. sending more .. etc etc until it's all done, then carrying on with the main program.
I've not used threads or callbacks or suchlike in this environment before but will learn - I just need a pointer to the most straigtforward ways to do it.
Thanks
Rob
The .NET serialport uses buffers, learn to work with them.
Sending packets that are (far) smaller than the Send-buffer can be done w/o threading.
Receiving can be done by the DataReceived event but beware that that is called from another thread. You might as well start your own thread and use blocking reads from there.
The best approach depends on what your 'packets' and protocol look like.
I think to have a long experience about serial comm, both MCU and PC-based.
I strongly UNSUGGEST the single-thread based solution, although it is very straigthful for light-speed testing, but absolutely out for final releases.
Surely you may choose among several patterns, but they are mostly shaped around a dedicated thread for the comm process and a finite-state-machine to parse the protocol (during receiveing).
The prevoius answers give you an idea to how build a simple program, but it might depends on the protocol specification, target device, scope of the application, etc.
there are of course different ways.
I will describe a thread based and an async operation based way:
If you don't use threads, your app will block as long as the operation is performing. This is not what a user is expecting today. Since you are talking about a series of sending and receiveing commands, I would recommend starting the protocol as a thread and then waiting for it to finish. You might also place an Abort button if neccesary. Set the ReadTimeout values and at every receive be ready to catch the exception! An introducing into creating such a work thread is here
If you want to, use Async Send/Receive functions instead of a thread (e.g. NetworkStream.BeginRead etc.). But this is more difficult because you have to manage state between the calls: I recommend using a Finite State Machine then. In fact you create an enumeration (i.e. ProtocolState) and change the state whenever an operation has completed. You can then simply create a function that performs the next step of the protocol with a simple switch/case statement. Since you are working with a remote entity (in your case the serial target system), you always have to consider the device is not working or stops working during the protocol. Do this by starting a timeout timer (e.g. set to 2000ms) and start it after sending each command (assuming each command will get a reply in your protocol). Stop it if the command was received successfully or on timeout.
You could also implement low-level handshaking on the serial port; set the serial port's Handshake property to rts/cts or xon/xoff.
Otherwise (or in addition), use a background worker thread. For simple threads, I like a Monitor.Wait/Pulse mechanism for managing the thread.
I have some code that does read-only serial communications in a thread; email me and I'll be happy to send it to you.
I wasn't sure from your question if you were designing both the PC and embedded sides of the communication link, if you are you might find this SO question interesting.