We Keep Coding

.net c# best way to wait for async events to complete and still have a "synchronous" flow in your code

My generalized question is this: how do you write asynchronous code that is still clear and easy to follow, like a synchronous solution would be?
My experience is that if you need to make some synchronous code asynchronous, using something like BackgroundWorker, you no longer have a series of easy to follow program statements that express your overall intent and order of activities, you end up instead with a bunch of "Done" Event Handlers, each of which starts the next BackgroundWorker, producing code that's really hard to follow.
I know that's not very clear; something more concrete:
Let's say a function in my WinForms application needs to start up some amazon EC2 instances, wait for them to become running, and then wait for them to all accept an SSH connection. A synchronous solution in pseudo code might look like this:
instances StartNewInstances() {
instances = StartInstances()
WaitForInstancesToBecomeRunning(instances)
WaitForInstancesToAcceptSSHConnection(instances).
return (instances)
}
That's nice. What is happening is very clear, and the order of program actions is very clear. No white noise to distract you from understanding the code and the flow. I'd really like to end up with code that looks like that.
But in reality, I can't have a synchronous solution .. each of those functions can run for a long time, and each needs to do things like: update the ui, monitor for time-outs being exceeded, and retry operations periodically until success or time-out. In short, each of these needs to be happening in the background so the foreground UI thread can continue on.
But if I use solutions like BackgroundWorker, it seems like I don't end up with nice easy to follow program logic like the above. Instead I might start a background worker from my UI thread to perform the first function, and then my ui thread goes back to the UI while the worker thread runs. When it finishes, its "done" event handler might start the next Background Worker. WHen it finishes, its "done" event handler might start the last BackgroundWorker, and so on. Meaning you have to "follow the trail" of the Done Event handlers in order to understand the overall program flow.
There has to be a better way that a) lets my UI thread be responsive, b) let's my async operations be able to update the ui and most importantly c) be able to express my program as series of consecutive steps (as I've shown above) so that someone can understand the resultant code
Any and all input would be greatly appreciated!
Michael

My generalized question is this: how do you write asynchronous code that is still clear and easy to follow, like a synchronous solution would be?
You wait for C# 5. It won't be long now. async/await rocks. You've really described the feature in the above sentence... See the Visual Studio async homepage for tutorials, the language spec, downloads etc.
At the moment, there really isn't a terribly clean way - which is why the feature was required in the first place. Asynchronous code very naturally becomes a mess, especially when you consider error handling etc.
Your code would be expressed as:
async Task<List<Instance>> StartNewInstances() {
List<Instance> instances = await StartInstancesAsync();
await instances.ForEachAsync(x => await instance.WaitUntilRunningAsync());
await instances.ForEachAsync(x => await instance.WaitToAcceptSSHConnectionAsync());
return instances;
}
That's assuming a little bit of extra work, such as an extension method on IEnumerable<T> with the form
public static Task ForEachAsync<T>(this IEnumerable<T> source,
Func<T, Task> taskStarter)
{
// Stuff. It's not terribly tricky :(
}

On the off chance that you can't wait for 5 as Jon rightly suggests, I'd suggest that you look at the Task Parallel Library (part of .NET 4). It provides a lot of the plumbing around the "Do this asynchronously, and when it finishes do that" paradigm that you describe in the question. It also has solid support for error handling in the asynchronous tasks themselves.

Async/await is really the best way to go. However, if you don't want to do wait, you can try Continuation-passing-style, or CPS. To do this, you pass a delegate into the async method, which is called when processing is complete. In my opinion, this is cleaner than having all of the extra events.
That will change this method signature
Foo GetFoo(Bar bar)
{
return new Foo(bar);
}
To
void GetFooAsync(Bar bar, Action<Foo> callback)
{
Foo foo = new Foo(bar);
callback(foo);
}
Then to use it, you would have
Bar bar = new Bar();
GetFooAsync(bar, GetFooAsyncCallback);
....
public void GetFooAsyncCallback(Foo foo)
{
//work with foo
}
This gets a little tricky when GetFoo could throw an exception. The method I prefer is to chage the signature of GetFooAsync.
void GetFooAsync(Bar bar, Action<Func<Foo>> callback)
{
Foo foo;
try
{
foo = new Foo(bar);
}
catch(Exception ex)
{
callback(() => {throw ex;});
return;
}
callback(() => foo);
}
Your callback method will look like this
public void GetFooAsyncCallback(Func<Foo> getFoo)
{
try
{
Foo foo = getFoo();
//work with foo
}
catch(Exception ex)
{
//handle exception
}
}
Other methods involve giving the callback two parameters, the actual result and an exception.
void GetFooAsync(Bar bar, Action<Foo, Exception> callback);
This relies on the callback checking for an exception, which could allow it to be ignored. Other methods have two call backs, one for success, and one for failure.
void GetFooAsync(Bar bar, Action<Foo> callback, Action<Exception> error);
To me this makes the flow more complicated, and still allows the Exception to be ignored.
However, giving the callback a method that must be called to get the result forces the callback to deal with the Exception.

When it finishes, its "done" event handler might start the next Background Worker.
This is something that I've been struggling with for a while. Basically waiting for a process to finish without locking the UI.
Instead of using a backgroundWorker to start a backgroundWorker however, you can just do all the tasks in one backgroundWorker. Inside the backgroundWorker.DoWork function, it runs synchronously on that thread. So you can have one DoWork function that processes all 3 items.
Then you have to wait on just the one BackgroundWorker.Completed and have "cleaner" code.
So you can end up with
BackgroundWorker_DoWork
returnValue = LongFunction1
returnValue2 = LongFunction2(returnValue)
LongFunction3
BackgroundWorker_ProgressReported
Common Update UI code for any of the 3 LongFunctions
BackgroundWorker_Completed
Notify user long process is done

In some scenario (will explain later), you can wrap the async calls to a method like the following pseudo code:
byte[] ReadTheFile() {
var buf = new byte[1000000];
var signal = new AutoResetEvent(false);
proxy.BeginReadAsync(..., data => {
data.FillBuffer(buf);
signal.Set();
});
signal.WaitOne();
return buf;
}
For the above code to work, the call back needs to be invoked from a different thread. So this depends on what you are working with. From my experience, at least Silverlight web service calls are handled in UI thread, which means the above pattern cannot be used - if the UI thread is blocked, the previous begin call even cannot be carried out. If you are working with this kind of frameworks, another way to handle multiple async calls is to move your higher level logic to a background thread and use UI thread for communication. However, this approach is a little bit over killing in most cases because it requires some boilerplate code to start and stop background thread.

Marshalling Events Across Threads

I imagine this may be marked as repetitious and closed, but I cannot for the life of me find a clear, concise answer to this question. All the replies and resources deal almost exclusively with Windows Forms and utilizing pre-built utility classes such as BackgroundWorker. I would very much like to understand this concept at its core, so I can apply the fundamental knowledge to other threading implementations.
A simple example of what I would like to achieve:
//timer running on a seperate thread and raising events at set intervals
//incomplete, but functional, except for the cross-thread event raising
class Timer
{
//how often the Alarm event is raised
float _alarmInterval;
//stopwatch to keep time
Stopwatch _stopwatch;
//this Thread used to repeatedly check for events to raise
Thread _timerThread;
//used to pause the timer
bool _paused;
//used to determine Alarm event raises
float _timeOfLastAlarm = 0;
//this is the event I want to raise on the Main Thread
public event EventHandler Alarm;
//Constructor
public Timer(float alarmInterval)
{
_alarmInterval = alarmInterval;
_stopwatch = new Stopwatch();
_timerThread = new Thread(new ThreadStart(Initiate));
}
//toggles the Timer
//do I need to marshall this data back and forth as well? or is the
//_paused boolean in a shared data pool that both threads can access?
public void Pause()
{
_paused = (!_paused);
}
//little Helper to start the Stopwatch and loop over the Main method
void Initiate()
{
_stopwatch.Start();
while (true) Main();
}
//checks for Alarm events
void Main()
{
if (_paused && _stopwatch.IsRunning) _stopwatch.Stop();
if (!_paused && !_stopwatch.IsRunning) _stopwatch.Start();
if (_stopwatch.Elapsed.TotalSeconds > _timeOfLastAlarm)
{
_timeOfLastAlarm = _stopwatch.Elapsed.Seconds;
RaiseAlarm();
}
}
}
Two questions here; primarily, how do i get the event to the main thread to alert the interested parties of the Alarm event.
Secondarily, regarding the Pause() method, which will be called by an object running on the main thread; can i directly manipulate the Stopwatch that was created on the background thread by calling _stopwatch.start()/_stopwatch.stop(). If not, can the main thread adjust the _paused boolean as illustrated above such that the background thread can then see the new value of _paused and use it?
I swear, I've done my research, but these (fundamental and critical) details have not made themselves clear to me yet.
Disclaimer: I am aware that there are classes available that will provide the exact particular functionality that I am describing in my Timer class. (In fact, I believe the class is called just that, Threading.Timer) However, my question is not an attempt to help me implement the Timer class itself, rather understand how to execute the concepts that drive it.

Note: im writing this here because theres not enough space on comments, this is of course not a complete, nor half a complete answer:
I've always used Events to signal unrelated code to do something, so that was how I described my intent. Forgive me though, I'm not sure I see the difference between marshaling and event versus marshaling another type of data (signal).
Conceptually both can be treated as events. The difference between using provided sync/signalining objects and trying to implement something like this by urself, is who and how gets the job done.
An event in .net is just a delegate, a list of pointers to methods that should be executed when the provider of the event fires it.
What youre talking about (marshalling the event), if i understand you correctly, is sharing the event object when something happens, while the concept of signalig usually talks about an object which is shared to start with, and both threads "know" something happened by checking its state either manualy or automatily (relying on provided tools by both .net and windows).
In the most basic scenario, you can implement such a signaling concept by using a boolean variable, with one thread constantly looping to check if the value of the boolean is true, and another setting to such, as a way to signal something happend. The different signaling tools provided by .NET do this in a less resource-wasting maner, by also not executing the waiting thread, as long as theres no signal (the boolean equals to false), but conceptually, it is the same idea.

You cannot magically execute code on an existing thread.
Instead, you need the existing thread to explicitly execute your code, using a thread-safe data structure to tell it what to do.
This is how Control.Invoke works (which is in turn how BackgroundWorker works).
WiinForms runs a message loop in Application.Run() which looks roughly like this:
while(true) {
var message = GetMessage(); //Windows API call
ProcessMessage(message);
}
Control.Invoke() sends a Windows message (using thread-safe message passing code within Windows) telling it to run your delegate. ProcessMessage (which executes on the UI thread) will catch that message and execute the delegate.
If you want to do this yourself, you will need to write your own loop. You can use the new thread-safe Producer-Consumer collections in .Net 4.0 for this, or you can use a delegate field (with Interlocked.CompareExchange) and an AutoResetEvent and do it yourself.

Cancellation of a Task without Providing the Task method with the CancellationTokenSource

I'm trying to provide a functionality of having two Methods one called StartTask(action mymethod)
and the other called StopTask();
problem is the action has to have access to the CancellationTokenSource to check for cancellation and exit the method (return) which is not really what i want the method could be in another component or layer , i cant push every Method to have access to that cancellationtokensource,
i cant push the designer/developer of the component which have the process method to check for cancellation and return.
is there is any way to have something like this , i know it sound strange and inapplicable , just thought of asking.
this is the best i got:
CancellationTokenSource cancellationTokenSource;
private void button1_Click(object sender, EventArgs e)
{
cancellationTokenSource = new CancellationTokenSource();
Task t = new Task(() => Dowork(CancellationAction), cancellationTokenSource.Token, TaskCreationOptions.LongRunning);
t.Start();
}
private bool CancellationAction()
{
if (cancellationTokenSource.IsCancellationRequested)
{
label1.Invoke(new MethodInvoker(() =>
{
label1.Text = "Cancellation Requested!";
}));
return true;
}
return false;
}
private void Dowork(Func<bool> Return)
{
int x = 1;
while (true)
{
x++;
label1.Invoke(new MethodInvoker(() =>
{
label1.Text = x.ToString();
}));
Thread.Sleep(1000);
if (Return())
{
return;
}
}
}
problem with this is DoWork now has to have one parameter which is func , but what if the method already takes other parameters ? the creation of task will be in another class which might not have any idea what parameters to pass beside CancellationAction

If the component does not provide a way to cancel one of its running tasks, then the caller should not be able to cancel it. It could leave the application/database/anything in an unknown state.
So basically the lower level component should provide the caller with a way to cancel a task (ManualResetEvent, CancelAsync method like the BackgroundWorker, etc.). Otherwise the caller should wait for it to finish.
If the lower level component does not provide such a feature, it is most of the time considered as bad design.

I'm not sure that I entirely understand your question, but I'll take a stab at it. It seems like you're trying to solve two problems at once here.
First you're trying to pass parameters to an asynchronous thread and/or cancel that thread (very similar issues). As others have stated BackgroundWorker already handles canceling. That implementation is similar to passing any argument to your thread. If I were replicating that functionality for instance I'd add a Cancel property or method to my worker thread that any other component could call and check a backing value in my main thread loop. No reason to do that for canceling threads these days, just an example of passing and using values to a worker thread.
The other problem that it looks like you need to solve is how to send messages between different parts of your application that shouldn't otherwise need to reference each other. Typically I've seen this done with a service provider of some sort. Implement an interface on a context or common model that all components receive an instance of or have easy access to. The interface should contain any events, methods and properties so the different components can communicate.
E.g. (probably a bad example but...) If my grammar checking routine should cancel when a document is closed, I would define a DocumentClosing event and OnDocumentClosing method on an IDocumentService interface and implement that interface in an appropriate context/model. When creating my document viewer UI component and grammar checker thread component I would inject an instance of the context/model typed as the interface. When the document viewer starts to close the document, it calls the OnDocumentClosing method from the interface. When the thread is created it would attach to the DocumentClosing event and if the event fires a flag is set. Then at intervals while checking grammar, I would check the flag and cancel as appropriate.
This sort of implementation gives you the flexibility to have any component trigger appropriate events and any other component react to them regardless of where in your application the components are used. In fact, this approach is useful even in synchronous situations such as menu items changing state in response to application events. It allows for easy unit testing of all your components. And the segregation of responsibility means that you can easily change any of the trigger points and responses as needed.

Why don't you use BackgroundWorkerThread or other threading mechanism?
Is there a particular reason for using Task Parallel Library?
BackgroundWorkerThread will give you a change to cancel the task and then respond to cancellation.

Best practice for an endless/ periodic execution of code in C#

Often in my code I start threads which basically look like this:
void WatchForSomething()
{
while(true)
{
if(SomeCondition)
{
//Raise Event to handle Condition
OnSomeCondition();
}
Sleep(100);
}
}
just to know if some condition is true or not (for example if a have a bad coded library with no events, just boolean variables and I need a "live-view" of them).
I wonder if there is a better way to accomplish this kind of work like a Windows function to hook in which can run my methods all x sec. Or should I code a global event for my app, raising all x secs and let him call my methods like this:
//Event from Windows or selfmade
TicEvent += new TicEventHandler(WatchForSomething));
and then this method:
void WatchForSomething()
{
if(SomeCondition)
{
//Raise Event to handle Condition
OnSomeCondition();
}
}
So, I hope this is not closed because of being a "subjective question" or something, I just want to know what the best practice for this kind of work is.

There isn't necessarily a "best way" to write long-running event processing code. It depends on what kind of application you are developing.
The first example you show is the idiomatic way in which you would often see the main method of a long-running thread written. While it's generally desirable to use a mutex or waitable event synchronization primitive rather than a call to Sleep() - it is otherwise a typical pattern used to implement event processing loops. The benefit of this approach is that it allows specialized processing to run on a separate thread - allowing your application's main thread to perform other tasks or remain responsive to user input. The downside of this approach is that it may require the use of memory barriers (such as locks) to ensure that shared resources are not corrupted. It also makes it more difficult to update your UI, since you must generally marshal such calls back to the UI thread.
The second approach is often used as well - particularly in systems that already have an event-drive API such as WinForms, WPF, or Silverlight. Using a timer object or Idle event is the typical manner in which periodic background checks can be made if there is no user-initiated event that triggers your processing. The benefit here is that it's easy to interact and update user interface objects (since they are directly accessible from the same thread) and it's mitigates the need for locks and mutexes to protected data. One potential downside of this approach is if the processing that must be performed is time-consuming it can make your application unresponsive to user input.
If you are not writing applications that have a user interface (such as services) then the first form is used much more often.
As an aside ... when possible, it's better to use a synchronization object like an EventWaitHandle or Semaphore to signal when work is available to be processed. This allows you to avoid using Thread.Sleep and/or Timer objects. It reduces the average latency between when work is available to be performed and when event processing code is triggered, and it minimizes the overhead of using background threads, since they can be more efficiently scheduled by the runtime environment and won't consume any CPU cycles until there's work to do.
It's also worth mentioning that if the processing you do is in response to communications with external sources (MessageQueues, HTTP, TCP, etc) you can use technologies like WCF to provide the skeleton of your event handling code. WCF provides base classes that make it substantially easier to implement both Client and Server systems that asynchronously respond to communication event activity.

If you have a look at Reactive Extensions, it provides an elegant way of doing this using the observable pattern.
var timer = Observable.Interval(TimeSpan.FromMilliseconds(100));
timer.Subscribe(tick => OnSomeCondition());
A nice thing about observables is the ability to compose and combine further observables from existing ones, and even use LINQ expressions to create new ones. For example, if you wanted to have a second timer that was in sync with the first, but only triggering every 1 second, you could say
var seconds = from tick in timer where tick % 10 == 0 select tick;
seconds.Subscribe(tick => OnSomeOtherCondition());

By the way, Thread.Sleep is probably never a good idea.
A basic problem with Thread.Sleep that people are usually not aware of, is that the internal implementation of Thread.Sleep does not pump STA messages. The best and easiest alternative, if you have to wait a given time and can't use a kernel sync object, is to replace Thread.Sleep with Thread.Join on the current thread, with the wanted timeout. Thread.Join will behave the same, i.e. the thread would wait the wanted time, but in the meantime STA objects will be pumped.
Why this is important (some detailed explanatiopn follows)?
Sometimes, without you even knowing, one of your threads may have created an STA COM object. (For example this sometimes happens behind the scenes when you use Shell APIs). Now suppose a thread of yours has created an STA COM object, and is now in a call to Thread.Sleep.
If at sometime the COM object has to be deleted (which can happen at an unexpected time by the GC), then the Finalizer thread will try calling the object's distruvtor. This call will be marshalled to the object's STA thread, which will be blocked.
Now, in fact, you will have a blocked Finalizer thread. In this situations objects can't be freed from memory, and bad things will follow.
So the bottom line: Thread.Sleep=bad. Thread.Join=reasonable alternative.

The first example you show is a rather inelegant way to implement a periodic timer. .NET has a number of timer objects that make this kind of thing almost trivial. Look into System.Windows.Forms.Timer, System.Timers.Timer and System.Threading.Timer.
For example, here's how you'd use a System.Threading.Timer to replace your first example:
System.Threading.Timer MyTimer = new System.Threading.Timer(CheckCondition, null, 100, 100);
void CheckCondition(object state)
{
if (SomeCondition())
{
OnSomeCondition();
}
}
That code will call CheckCondition every 100 milliseconds (or thereabouts).

You don't provide a lot of background on why you're doing this, or what you're trying to accomplish, but if its possible, you might want to look into creating a windows service.

Use a BackgroundWoker for additional thread safe measures:
BackgroundWorker bw = new BackgroundWorker();
bw.WorkerSupportsCancellation = true;
bw.WorkerReportsProgress = true;
.
.
.
private void bw_DoWork(object sender, DoWorkEventArgs e)
{
BackgroundWorker worker = sender as BackgroundWorker;
for (;;)
{
if (worker.CancellationPending == true)
{
e.Cancel = true;
break;
}
else
{
// Perform a time consuming operation and report progress.
System.Threading.Thread.Sleep(100);
}
}
}
For more info visit: http://msdn.microsoft.com/en-us/library/cc221403%28v=vs.95%29.aspx

A very simple way for non blocking wait other threads/tasks is:
(new ManualResetEvent(false)).WaitOne(500); //Waits 500ms

Using VerifyAccess and CheckAccess methods of DispatcherObject

While going through this article I came across this statement -
If you are writing your own WPF
objects, such as controls, all methods
you use should call VerifyAccess
before they perform any work. This
guarantees that your objects are only
used on the UI thread, like this
//Using VerifyAccess and CheckAccess
public class MyWpfObject : DispatcherObject
{
public void DoSomething()
{
VerifyAccess();
// Do some work
}
public void DoSomethingElse()
{
if (CheckAccess())
{
// Something, only if called
// on the right thread
}
}
}
I haven't seen this in any of the custom controls I have come across(as far as I remember).
Do you use this while building custom controls?
Is it must to do this or just nice to have?
Anyone ever faced any issue due to absence of this in your controls?

Nah, never used this. And never noticed somebody use it in the context of Custom Controls. This rule is not followed in WPF Toolkit too.
This approach not only pollutes the code but also makes your custom control responsible for something it shouldn't care about. Consider situation where you always doing:
// Don't do this in all methods of your custom control!
public void Foo()
{
if (!CheckAccess())
{
Dispatcher.Invoke(()=> Foo()); // Transit to UI Thread
return;
}
// .. do work in UI.
}
At first glance this code looks fine. If you are not in UI thread, transit to UI thread, perform operation and return result. Right? - WRONG!
Problem 1. When you call Dispatcher.Invoke() you block calling thread until your request is processed by UI thread. This leads to poor performance. Of course, you can change this to Dispatcher.BeginInvoke() now your clients should be aware that their operation is done asynchronously. I.e. if client writes something to control, and then immediately reads it back there is no guarantee, that the operation already executed by UI thread.
Problem 2. Consider subsequent calls to the method Foo() from non UI thread. For example it's called in cycle:
// Somewhere not in UI
for (int i = 0; i < 1000000; i++)
{
control.Foo(); // Looks good, but performance is awful!
}
Instead of blocking calling thread 1000000 times, developer could implement one check in the calling thread and transit to UI when necessary, instead of unconsciously jumping back and worth between threads.
Furthermore WPF will make this check for you when you access UI element from non-UI thread. It screams loud enough to crush application and be heard by developer who has done something wrong :).
Hope this helps.