We Keep Coding

Event handler timing and threading

I am still learning C# so please be easy on me. I am thinking about my application I am working on and I can't seem to figure out the best approach. This is not a forms application but rather a console. I am listening to a UDP port. I get UDP messages as fast as 10 times per second. I then look for a trigger in the UDP message. I am using an event handler that is raised each time i get a new UDP packet which will then call methods to parse the packet and look for my trigger. So, i have these questions.
With regard to threading, I assume a thread like my thread that listens to the UDP data should be a permanent thread?
Also on threading, when I get my trigger and decide to do something, in this case send a message out, i gather that I should use a thread pool each time I want to perform this task?
On thread pools, I am reading that they are not very high priority, is that true? If the message I need to send out is critical, can i rely on thread pools?
With the event handler which is raised when i get a UDP packet and then calls methods, what is the best way to ensure my methods all complete before the next packet/event is raised? At times I see event queue problems because if any of the methods take a bit longer than they should (for exampe writing to a DB) and the next packet comes in 100ms later, you get event queue growth because you cannot consume events in a timely manner. Is there a good way to address this?

With regard to threading, I assume a thread like my thread that listens to the UDP data should be a permanent thread?
There are no permanent threads. However there should be a thread that is responsible for receiving. Once you start it, let it run until you no longer need to receive any messages.
Also on threading, when I get my trigger and decide to do something, in this case send a message out, i gather that I should use a thread pool each time I want to perform this task?
That depends on how often would you send out messages. If your situation is more like consumer/producer than a separate thread for sending is a good idea. But if you send out a message only rarely, you can use thread pool. I can't define how often rare means in this case, you should watch your app and decide.
On thread pools, I am reading that they are not very high priority, is that true? If the message I need to send out is critical, can i rely on thread pools?
You can, it's more like your message will be delayed because of slow message processing or slow network rather than the thread pool.
With the event handler which is raised when i get a UDP packet and then calls methods, what is the best way to ensure my methods all complete before the next packet/event is raised? At times I see event queue problems because if any of the methods take a bit longer than they should (for exampe writing to a DB) and the next packet comes in 100ms later, you get event queue growth because you cannot consume events in a timely manner. Is there a good way to address this?
Queue is a perfect solution. You can have more queues if some messages are independent of others and their execution won't collide and then execute them in parallel.

I'll adress your points:
your listeting thread must be a 'permanent' thread that gets messages and distribute them.
(2+3) - Look at the TPL libarary you should use it instead of working with threads and thread pools (unless you need some fine control over the operations which, from your question, seems like you dont need) - as MSDN states:
The Task Parallel Library (TPL) is based on the concept of a task, which represents an asynchronous operation. In some ways, a task resembles a thread or ThreadPool work item, but at a higher level of abstraction
Look into using MessageQueues since what you need is a place to receive messages, store them for some time (in memory in your case)and handle them at your own pace.
You could implement this yourself but you'll find it gets complicated quickly,
I recommend looking into NetMQ - it's easy to use, especially for what you describe, and it's in c#.

Wait for COM Event to complete

How do I block the UI thread while waiting for COM event to complete. I subscribe to update event of COM which signals the event has completed.
MyRData.OnUpdate += OnUpdate;
I do not own the COM code and cannot make changes to it.
I tried AutoResetEvent however that blocks the UI thread and i dont recieve updates from COM.

My answer is very similar to #EricBrown's one, but there is one different point.
Creating a nested message loop with MsgWaitForMultipleObjectsEx may lead to code reentrancy on the same thread (via a window message dispatched by the inner PeekMessage/TranslateMessage/DispatchMessage pattern). At worst scenario, you may end up calling the same COM object method before the previous call has returned.
I would first try using CoWaitForMultipleHandles with COWAIT_DISPATCH_CALLS (but without COWAIT_DISPATCH_WINDOW_MESSAGES). In case your COM object is provided by an out-of-proc server, this most likely should work. Otherwise, you should consider putting some reentrancy checks in place.
I have a related question with some code showing how it could be done with C# (I had to use COWAIT_DISPATCH_WINDOW_MESSAGES there, otherwise the event I was after wasn't getting fired).
[UPDATE] Ideally, you should use async/await pattern for things like that and wrap your event as a task (e.g. here's how). I understand, sometimes it is not feasible to re-factor existing code to use this approach. However, if a pending operation takes considerable time to complete, a more user-friendly way to wait for its completion event might be just to show a modal dialog with a nice "please wait..." message (as discussed here in comments). You'd just close this dialog from your event handler. In fact, AFAIK, this is the only endorsed way for a WinForms app to enter a nested message loop.
[UPDATE] As Eric pointed out in comments, COWAIT_DISPATCH_WINDOW_MESSAGES is indeeded required for an STA thread. Apparently, COWAIT_DISPATCH_CALLS is intended for the new little-known ASTA model and has no meaning in other apartment types.
In case with out-of-proc COM servers, .NET event handlers are called back as free-threaded objects regardless of the waiting thread's apartment model (in my experience, it's never the same STA thread on which the out-of-proc object was originally created). Thus, waiting with WaitHandle.WaitOne (no pumping) should be sufficient. However, if the event handler accesses any state data besides the WaitHandle, proper synchronization is required (with locks etc).

Most likely you want to pump messages while waiting for an event. For this, MsgWaitForMultipleObjectsEx is invaluable. I have an answer (to a different question) that demonstrates a common usage pattern for MsgWaitForMultipleObjectsEx.

I finally ended up using
Application.DoEvents()

C# Event Handling Order

The scenario is an event on a buffer, that informs interested classes when there is data available to be collected. The event is fired as soon as new data is written to the buffer. When this is fired, the delegate for the event (in the interested class) starts reading data from the buffer.
My question is, if the event were fired again (before the method had finished reading all the data from the buffer) would the reading method be 'reset' or would the event wait for the method to finish reading the data before calling it again?

The event could only be fired again before the method had finished reading if it were fired on another thread. The event handlers will then (by default) be called again in that separate thread. There's no concept of an existing method being "reset", nor would it wait for the already-running handlers to finish before firing again.
Of course, you could potentially change how the handlers work, or how the event is fired - perhaps ensuring that the event handlers are only called from a single thread, with some sort of queue of events. It's impossible for us to tell whether that's appropriate for your situation though.

Neither, it would execute it alongside (in parallel), if on separate threads - otherwise execution would be blocking anyway.

Unless you've put Application.DoEvents() in your code (which is a horrible thing to do) then your event won't be interrupted.
In a multithreading scenario, there's the possibility of them running in parallel. I don't use multiple threads and events both at the same time, so I can't really say much about that, but it seems like Jon's covered that one nicely with his answer.

Raise event in one thread to invoke methods in a second thread

I'm working on a program which reacts to events coming from an internet socket, and possibly from timers as well. It seems natural to use two threads:
One for the main program
A second one which listens to the socket, parses the input, and raises an appropriate event.
Additional requirements:
The application should not rely on a UI thread (it may be run as a console application).
The main program should process messages synchronously, i.e. in the order in which they arrived.
The main thread must not block on waiting for timers (I guess this means I have to run timers on different threads).
And now for some questions :-):
I'm guessing requirement #1 means that I don't have a built-in message pump, so I can't use Invoke() from the socket listener / timer threads. Is this correct?
How can I safely raise events on one thread (e.g. the listener), and have the subscribers run synchronously on another (the main thread)?
It is very likely that new events will be raised before the subsequent handler is done. What will happen in this case? Will the event be buffed somewhere by the CLR, or will it be ignored?
And last but not least: I guess I'm aiming for the parallel for the message Producer/Consumer paradigm, but instead of messages, I want to use events. Do you think there is a better approach?
Thanks,
Boaz
EDIT
I want to explain my motivation for using events in the first place. The application is an automated trading engine which has to respond to events that happen in the market (e.g. a change in the price of a stock). When this happens, there may be multiple subscribers on the main thread which should be invoked, which is a classical scenario to use events.
I guess I can always use the Producer/Consumer with some message queue, and have the consumer raise events on the main thread, but I figured there might be a more direct way.

I think using messages will be the simplest way. If you are using C# 4 this is very easy thanks to the BlockingCollection<>
So have a shared BlockingCollection, where Message is your message class.
Then in your worker thread you do this
var msgEnum = blockingCollection.GetConsumingEnumerable();
//Per thread
foreach( Message message in msgEnum )
{
//Process messages here
}
That is it.
The GetConsumingEnumerable() will block until there is a message to process. It will then remove the message from the queue and your loop will process it.
What is nice about this is that you can add more threads and in each one you just have the foreach loop.
When you are done call blockingCollection.CompletedAdding();
BTW the queue handles concurrency and will queue messages sent at the same time etc.
Hope this helps
Andre

You could implement a shared queue between your threads. Whenever an event is raised you could push it in the queue. The main thread is an endless loop that checks for new events, removes them from the queue, handles the event and when there are no more events it sleeps for some time.

BackgroundWorker might be causing my application to hang

I have a Form that uses a BackgroundWorker to execute a series of tests. I use the ProgressChanged event to send messages to the main thread, which then does all of the updates on the UI. I've combed through my code to make sure I'm not doing anything to the UI in the background worker. There are no while loops in my code and the BackgroundWorker has a finite execution time (measured in seconds or minutes). However, for some reason when I lock my computer, often times the application will be hung when I log back in. The thing is, the BackgroundWorker isn't even running when this happens. The reason I believe it is related to the BackgroundWorker though is because the form only hangs when the BackgroundWorker has been executed since the application was loaded (it only runs when given a certain user input).
I pass this thread a List of TreeNodes from a TreeView in my UI through the RunWorkerAsync method, but I only read those nodes in the worker thread..any modifications I make to them is done in the UI thread through the progressChanged event.
I do use Thread.Sleep in my worker thread to execute tests at timed intervals (which involves sending messages over a TCP socket, which was not created in the worker thread).
I am completely perplexed as to why my application might be hanging. I'm sure I'm doing something 'illegal' somewhere, I just don't know what.

I pass this thread a List of TreeNodes from a TreeView in my UI through the RunWorkerAsync method, but I only read those nodes in the worker thread.
By "only read" I assume you mean "only access property getters". But property getters can execute code that you don't control - for example TreeNode.IsSelected will call a native method and send a Windows message (take a look with Reflector).
Instead you should extract the data you need from the TreeView in the UI thread and pass it to the background worker. Not only will you avoid this problem, but your design will be more loosely coupled.

Well, this one is old but it turned out that the problem was completely unrelated to my code. Due to recent changes in our software, the amount of logging had increased exponentially and our log buffer was overflowing causes the application to crash. It was just a coincidence that this was happening at the same time that I was working on this specific piece of code. In any case, I still made sure that I wasn't doing any operations on UI elements from a BackgroundWorker, even if it was as trivial as checking/unchecking a TreeNode.

Sounds like the dreaded UserPreferenceChanged event problem where a ui component has been created on a background thread without a message pump. The main ui thread synchronously sends the event to all registered ui windows and will hang because the ui component on the background worker thread is unable to process the UserPreferenceChanged event.