I am seeking advice on a project I have been assigned and I'm looking to see how it's done "professionally," or any suggestions that can lead me to the right direction.
I have a server piece that accepts commands from clients and pushes out byte streams to a serial port. Although multiple clients can send commands to this server piece, our hardware only can handle one command at a time. My problem is with queueing in the software end.
I've implemented a Queue<T> helper class that also inserts data into a DataSet containing: the requesting client number, message data (byte array to write to serial port) and message type (command description). It will also list the queue commands in a DataGrid (on the Form). Probably not the way to go, but that's the only thing I can think of as far as retaining the requesting client and the data and showing, visually, the queue.
Where do I handle the processing of the queue? I thought about handling it on a custom event where if the DataGrid list changed (item added/removed), grab the first row of data in the DataSet and send it out to the serial port.
Any comments or suggestions are greatly appreciated.
Thanks.
Edit: I forgot to add that it does require a response from the SerialPort as well, in order for the current executed command to be removed from the queue.
I would use a database table to store the queue of commands. The web app would add records to the queue and display the queue, then a separate process (such as a Windows service or console app) would request the next command from the database and send it to the serial port.
Client requests can come in at any time, they'll probably be handled by some proxy class (WCF?) on its own thread/task. Then that thread/ task needs to coordinate with the task that's 'inside' the model actually processing the requests.
A good class to do this with is the BlockingCollection.
The server-thread will block until there's something in the collection to work on. It can then take it from the collection in a thread safe manner and process it. Doing it this way ensures that the requests can be accepted when they arrive, but they are processed on at a time.
The overall pattern to think of here is producer-consumer.
GJ
If it is a high trasaction web application you might want to look at queueing system such as MSMQ, Service Broker Queue or RabbitMQ. Window service can then pick up the queued items and send it to serial port.
Related
We are running multiple instances of a windows service that reads messages from a Topic, runs a report, then converts the results into a PDF and emails them to a user. In case of exceptions we simply log the exception and move on.
The use case we want to handle is when the service is shut down we want to preserve the jobs that are currently running so they can be reprocessed by another instance of the service or when the service is restarted.
Is there a way of requeueing a message? The hacky solution would be to just republish the message from the consuming service, but there must be another way.
When incoming messages are processed, their data is put in an internal queue structure (not a message queue) and processed in batches of parallel threads, so the IbmMq transaction stuff seems hard to implement. Is that what I should be using though?
Your requirement seems to be hard to implement if you don't get rid of the "internal queue structure (not a message queue)" if this is not based on a transaction oriented middleware. The MQ queue / topic works well for multi-threaded consumers, so it is not apparent what you gain from this intermediate step of moving the data to just another queue. If you start your transaction with consuming the message from MQ, you can have it being rolled back when something goes wrong.
If I understood your use case correctly, you can use Durable subscriptions:
Durable subscriptions continue to exist when a subscribing application's connection to the queue manager is closed.
The details are explained in DEFINE SUB (create a durable subscription). Example:
DEFINE QLOCAL(THE.REPORTING.QUEUE) REPLACE DEFPSIST(YES)
DEFINE TOPIC(THE.REPORTING.TOPIC) REPLACE +
TOPICSTR('/Path/To/My/Interesting/Thing') DEFPSIST(YES) DURSUB(YES)
DEFINE SUB(THE.REPORTING.SUB) REPLACE +
TOPICOBJ(THE.REPORTING.TOPIC) DEST(THE.REPORTING.QUEUE)
Your service instances can consume now from THE.REPORTING.QUEUE.
While I readily admit that my knowledge is shaky, from what I understood from IBM’s [sketchy, inadequate, obtuse] documentation there really is no good built in solution. With transactions the Queue Manager assumes all is well unless it receives a roll back request and when it does it rolls back to a syncpoint, so if you’re trying to roll back to one message but two other messages have completed in the meantime it will roll back all three.
We ended up coding our own solution updating the way we’re logging messages and marking them as completed in the DB. Then on both startup and shutdown we find the uncompleted messages and programmatically publish them back to the queue, limiting the DB search by machine name so if we have multiple instances of the service running they won’t duplicate message processing.
I am currently writing a C# Program and am struggling a bit with a conceptional question.
A domain of the application is the communication with a Microcontroller via UDP. Therefore I am using the UDPClient and wrote a 'communicator' which does some encoding/decoding, checksum checks, etc.
I need to have some kind of Controller which allows me to send commands to the UDP-Server (the device). Some of them are a single pair of Send/Receive, others are long-running.
I thought about the following implemenation:
My idea was that every capability of my Microcontroller/UDP-Server is abstracted in a class, which has an 'Execute'-function which subscribes to the 'Receive' of the underlying Communicator and sends it request.
However, I am a bit stuck here: I could wait in the 'Execute ' function of each task(capability) for an event which I trigger in the OnReceive-Handler (or timeout). But that would not work if I one task needs multiple Send/Receive.
So, is there any good Design-Pattern for that? Or does anyone have a good advise on how to implement it in a proper way?
You can have a Communicator and a CommandExecuter and share a queue between them. Communicator puts every command it receives from UDP and goes back to it's duty to receive new commands. CommandExecuter is always watching the queue. As long as there is a new command in the queue, takes it and executes it.
I'm currently working on an application that communicates with an electronic device via the SerialPort.
This communication is done in a half-duplex fashion, where the application is the master and the device is the slave. The master needs to send a message to the device, and the device needs to respond before the next message is sent.
If the message doesn't receive a response, the application needs to resend it.
The content of the next message is dependent on the result of the current message. i.e. each new message has an incremented sequence number, and sometimes data for the next message is taken from the reponse of the current one.
To send messages I use an interface to System.IO.Ports.SerialPort. When messages are received a SerialDataReceivedEventHandler is fired.
What's the best way for me to manage this? Is there a pattern that I can base this on?
I have worked on something similar in the past. This is the basic structure of the messaging I have used:
Application: Send Command Seq #1 -->
<-- Device: Acknowledge command, seq #1 (including any command-specific response)
If the device didn't acknowledge within 1 second, the same command would be resent, same sequence number. This "retry" sequence would happen 3 times and then it would time out, and retire the command.
The sequence number from the application side would increment by one for every command it sent that was not a retry. It would loop back to sequence number 1 after hitting 99.
The application will have to keep track of its state based on what command(s) is "in-flight" between it and the device, and what kind of response it has received. It can identify the response to a specific command by the sequence number the device puts in its acknowledgement.
To keep it simple to transition from state to state you can make it so that there is only ever one active command and don't move on until that one has been ack'ed or timed out and retired.
Shane Wealti approach is right but I think what you might be really asking yourself, should you use threads? Even though it is a master/slave scenario, should you have a listening thread? When do you listen to the port and when do you send?
The simplest approach - no threads
You don't need to use threads in this scenario due to master/slave configuration, all you have is two functions.
SendCommand(char * bfr)
{
}
RecieveCommand(char * bfr)
{
}
SendCommand( txBfr );
RecieveData( rxBfr );
// process receive buffer, prepare new command
SendCommand( txbfr );
RecieveCommand( rxBfr );
// and so on
The approach is the most simple one and totally functional. However since there are no threads and say your RecieveData() times out in one second, you GUI will not be responsive in that second. Note that you are not listening to the port all the time but only when are expecting a reply.
I might edit this to add Comprehensive approach using threads later but don't have the time right now.
I can use some help for a designing my COMport connection classes.
I have a device with a microcontroller (which I programmed) connected to my comport.
Whenever I send a message to it, it should send an acknowledge.
So whenever I send something over the comport, it should wait for an acknowledge before continuing.
Ofcourse, I dont want something like a while(!ack) wait().
When a ack is not received in 1 second or so, it should send the command again.
The Ack looks different for each type of command (note: only the type, no message-specific id).
The connected device also sends messages (apart from ACKs), which need to be handled by the application itself.
Does anybody has suggestions about an easy and flexible way (a design pattern maybe? a sample?) to fix this?
You'll probably want a dedicated thread that handles the communications. You'll need a Queue on which the client code can push a message, protect it with a ReaderWriterLockSlim. No need for the DataReceived event, just call SerialPort.Read() directly. Detect timeouts with the ReadTimeout property. If you get responses that need to go back to the client code then use an event.
Watch out designing the protocol, it isn't that easy to get right. You'll protect against loss of bytes with your scheme, but it is just as likely for the ACK to be lost. The microcontroller will see the same command twice. Now you need a "message number" to suppress duplicates and a way for both ends to synchronize them. Take a look at RFC 916.
I'm an embedded programmer trying to do a little bit of coding for a communications app and need a quick start guide on the best / easiest way to do something.
I'm successfully sending serial data packets but need to impliment some form of send/ response protocol to avoid overflow on the target system and to ensure that the packet was received ok.
Right now - I have all the transmit code under a button click and it sends the whole lot without any control.
What's the best way to structure this code , i.e sending some packets - waiting for response .. sending more .. etc etc until it's all done, then carrying on with the main program.
I've not used threads or callbacks or suchlike in this environment before but will learn - I just need a pointer to the most straigtforward ways to do it.
Thanks
Rob
The .NET serialport uses buffers, learn to work with them.
Sending packets that are (far) smaller than the Send-buffer can be done w/o threading.
Receiving can be done by the DataReceived event but beware that that is called from another thread. You might as well start your own thread and use blocking reads from there.
The best approach depends on what your 'packets' and protocol look like.
I think to have a long experience about serial comm, both MCU and PC-based.
I strongly UNSUGGEST the single-thread based solution, although it is very straigthful for light-speed testing, but absolutely out for final releases.
Surely you may choose among several patterns, but they are mostly shaped around a dedicated thread for the comm process and a finite-state-machine to parse the protocol (during receiveing).
The prevoius answers give you an idea to how build a simple program, but it might depends on the protocol specification, target device, scope of the application, etc.
there are of course different ways.
I will describe a thread based and an async operation based way:
If you don't use threads, your app will block as long as the operation is performing. This is not what a user is expecting today. Since you are talking about a series of sending and receiveing commands, I would recommend starting the protocol as a thread and then waiting for it to finish. You might also place an Abort button if neccesary. Set the ReadTimeout values and at every receive be ready to catch the exception! An introducing into creating such a work thread is here
If you want to, use Async Send/Receive functions instead of a thread (e.g. NetworkStream.BeginRead etc.). But this is more difficult because you have to manage state between the calls: I recommend using a Finite State Machine then. In fact you create an enumeration (i.e. ProtocolState) and change the state whenever an operation has completed. You can then simply create a function that performs the next step of the protocol with a simple switch/case statement. Since you are working with a remote entity (in your case the serial target system), you always have to consider the device is not working or stops working during the protocol. Do this by starting a timeout timer (e.g. set to 2000ms) and start it after sending each command (assuming each command will get a reply in your protocol). Stop it if the command was received successfully or on timeout.
You could also implement low-level handshaking on the serial port; set the serial port's Handshake property to rts/cts or xon/xoff.
Otherwise (or in addition), use a background worker thread. For simple threads, I like a Monitor.Wait/Pulse mechanism for managing the thread.
I have some code that does read-only serial communications in a thread; email me and I'll be happy to send it to you.
I wasn't sure from your question if you were designing both the PC and embedded sides of the communication link, if you are you might find this SO question interesting.