We Keep Coding

C# await on a List<T> Count

I am upgrading some legacy WinForms code and I am trying to figure out what the "right way" as of .NET 4.6.1 to refactor the following.
The current code is doing a tight while(true) loop while checking a bool property. This property puts a lock() on a generic List<T> and then returns true if it has no items (list.Count == 0).
The loop has the dreaded Application.DoEvents() in it to make sure the message pump continues processing, otherwise it would lock up the application.
Clearly, this needs to go.
My confusion is how to start a basic refactoring where it still can check the queue length, while executing on a thread and not blowing out the CPU for no reason. A delay between checks here is fine, even a "long" one like 100ms+.
I was going to go with an approach that makes the method async and lets a Task run to do the check:
await Task.Run(() => KeepCheckingTheQueue());
Of course, this keeps me in the situation of the method needing to ... loop to check the state of the queue.
Between the waiting, awaiting, and various other methods that can be used to move this stuff to the thread pool ... any suggestion on how best to handle this?

What I need is how to best "poll' a boolean member (or property) while freeing the UI, without the DoEvents().
The answer you're asking for:
private async Task WaitUntilAsync(Func<bool> func)
{
while (!func())
await Task.Delay(100);
}
await WaitUntilAsync(() => list.Count == 0);
However, polling like this is a really poor approach. If you can describe the actual problem your code is solving, then you can get better solutions.
For example, if the list represents some queue of work, and your code is wanting to asynchronously wait until it's done, then this can be better coded using an explicit signal (e.g., TaskCompletionSource<T>) or a true producer/consumer queue (e.g., TPL Dataflow).

It's generally never a good idea for client code to worry about locking a collection (or sprinkling your code with lock() blocks everywhere) before querying it. Best to encapsulate that complexity out.
Instead I recommend using one of the .NET concurrent collections such as ConcurrentBag. No need for creating a Task which is somewhat expensive.
If your collection does not change much you might want to consider one of the immutable thread-safe collections such as ImmutableList<>.
EDIT: Upon reading your comments I suggest you use a WinForms Timer; OnApplicationIdle or BackgroundWorker. The problem with async is that you still need to periodically call it. Using a timer or app idle callback offers the benefit of using the GUI thread.

Depending on the use case, you could start a background thread or a background worker. Or maybe even a timer.
Those are executed in a different thread, and are therefore not locking the execution of your other form related code. Invoke the original thread if you have to perform actions on the UI thread.
I would also recommend to prevent locking as much as possible, for example by doing a check before actually locking:
if (list.Count == 0)
{
lock (lockObject)
{
if (list.Count == 0)
{
// execute your code here
}
}
}
That way you are only locking if you really need to and you avoid unnecessary blocking of your application.

I think what you're after here is the ability to await Task.Yield().
class TheThing {
private readonly List<int> _myList = new List<int>();
public async Task WaitForItToNotBeEmpty() {
bool hadItems;
do {
await Task.Yield();
lock (_myList) // Other answers have touched upon this locking concern
hadItems = _myList.Count != 0;
} while (!hadItems);
}
// ...
}

Tricky way of doing foreach in C#

I will first provide the pseudocode and describe it below:
public void RunUntilEmpty(List<Job> jobs)
{
while (jobs.Any()) // the list "jobs" will be modified during the execution
{
List<Job> childJobs = new List<Job>();
Parallel.ForEach(jobs, job => // this will be done in parallel
{
List<Job> newJobs = job.Do(); // after a job is done, it may return new jobs to do
lock (childJobs)
childJobs.AddRange(newJobs); // I would like to add those jobs to the "pool"
});
jobs = childJobs;
}
}
As you can see, I am performing a unique type of foreach. The source, the set (jobs), can simply be enhanced during the execution and this behaviour cannot be determined earlier. When the method Do() is called on an object (here, job), it may return new jobs to perform and thus would enhance the source (jobs).
I could call this method (RunUntilEmpty) recursively, but unfortunately the stack can be really huge and is likely to result in an overflow.
Could you please tell me how to achieve this? Is there a way of doing this kind of actions in C#?

If I understand correctly, you basically start out with some collection of Job objects, each representing some task which can itself create one or more new Job objects as a result of performing its task.
Your updated code example looks like it will basically accomplish this. But note that, as commenter CommuSoft points out, it won't make most efficient use of your CPU cores. Because you are only updating the list of jobs after each group of jobs has completed, there's no way for newly-generated jobs to run until all of the previously-generated jobs have completed.
A better implementation would use a single queue of jobs, continually retrieving new Job objects for execution as old ones complete.
I agree that TPL Dataflow may be a useful way to implement this. However, depending on your needs, you might find it simple enough to just queue the tasks directly to the thread pool and use CountdownEvent to track the progress of the work so that your RunUntilEmpty() method knows when to return.
Without a good, minimal, complete code example, it's impossible to provide an answer that includes a similarly complete code example. But hopefully the below snippet illustrates the basic idea well enough:
public void RunUntilEmpty(List<Job> jobs)
{
CountdownEvent countdown = new CountdownEvent(1);
QueueJobs(jobs, countdown);
countdown.Signal();
countdown.Wait();
}
private static void QueueJobs(List<Job> jobs, CountdownEvent countdown)
{
foreach (Job job in jobs)
{
countdown.AddCount(1);
Task.Run(() =>
{
// after a job is done, it may return new jobs to do
QueueJobs(job.Do(), countdown);
countdown.Signal();
});
}
}
The basic idea is to queue a new task for each Job object, incrementing the counter of the CountdownEvent for each task that is queued. The tasks themselves do three things:
Run the Do() method,
Queue any new tasks, using the QueueJobs() method so that the CountdownEvent object's counter is incremented accordingly, and
Signal the CountdownEvent, decrementing its counter for the current task
The main RunUntilEmpty() signals the CountdownEvent to account for the single count it contributed to the object's counter when it created it, and then waits for the counter to reach zero.
Note that the calls to QueueJobs() are not recursive. The QueueJobs() method is not called by itself, but rather by the anonymous method declared within it, which is itself also not called by QueueJobs(). So there is no stack-overflow issue here.
The key feature in the above is that tasks are continuously queued as they become known, i.e. as they are returned by the previously-executed Do() method calls. Thus, the available CPU cores are kept busy by the thread pool, at least to the extent that any completed Do() method has in fact returned any new Job object to run. This addresses the main problem with the version of the code you've included in your question.

Queue function calls to write char for char

What should it do
I'm trying to write some text char for char into a TextBlock. I'm using this code for this:
void WriteTextCharForChar(String text)
{
new Thread(() =>
{
foreach(Char c in text)
{
TxtDisplayAppendText(c.ToString());
Thread.Sleep(rnd.Next(20, 100));
}
Thread.CurrentThread.Abort();
}).Start();
}
The problem
The Problem is that the text, of course, gets mixed when calling this method more then once. What I would need is a kind of queue or wait any how until the currently text is written to the TextBlock.
Of curse I'm open to any other solution to get this working. Thank you!

So there are several options here.
You could add a lock around the work that the thread does, so that there is never more than one running at a time.
This has several problems though:
The items aren't necessarily processed in order
You're creating lots of threads, all of which are spending almost all of their time waiting; this is very wasteful of resources.
You could create a thread safe queue (BlockingCollection would be best) and then have a single thread reading from it and writing out the results while the UI thread just adds to the queue
This also has problems. Most notably you're creating a new thread that's going to be spending basically all of its time waiting around. This is probably better than #1, but not by a lot.
You could avoid using multiple threads entirely and do everything asynchronously. The Task Parallel Library gives you a lot of tools to help with this. This is the best option as it results in the creation of 0 extra threads.
So first we'll create a helper method that will handling the writing of the text itself, so that another thread can be the one to handle "scheduling" these calls. This method will be much easier to write using await. A key point to note is that, rather than blocking the current thread for an unknown period of time, it will use Task.Delay to continue execution at a point in the future without blocking the thread:
private async Task WriteText(string text)
{
foreach (char c in text)
{
TxtDisplayAppendText(c.ToString());
await Task.Delay(rnd.Next(20, 100));
}
}
Now for your method. We can manage our queue though what will be in effect a linked list of tasks. If we have a single field of type Task representing the "previous task" we can have each method call add a continuation to that task, and then set itself as the previous task. The next task will set itself as the continuation of that, and so on. Each continuation will fire when the previous task runs, or will run immediately if the previous task has already finished, so this gives us our "queue", effectively. Note that since WriteTextCharForChar is being called from the UI thread these calls are all already synchronized, so there's no need to lock around the manipulation of this task.
private Task previousWrite = Task.FromResult(false); //already completed task
private void WriteTextCharForChar(String text)
{
previousWrite = previousWrite.ContinueWith(t => WriteText(text))
.Unwrap();
}

.net c# best way to wait for async events to complete and still have a "synchronous" flow in your code

My generalized question is this: how do you write asynchronous code that is still clear and easy to follow, like a synchronous solution would be?
My experience is that if you need to make some synchronous code asynchronous, using something like BackgroundWorker, you no longer have a series of easy to follow program statements that express your overall intent and order of activities, you end up instead with a bunch of "Done" Event Handlers, each of which starts the next BackgroundWorker, producing code that's really hard to follow.
I know that's not very clear; something more concrete:
Let's say a function in my WinForms application needs to start up some amazon EC2 instances, wait for them to become running, and then wait for them to all accept an SSH connection. A synchronous solution in pseudo code might look like this:
instances StartNewInstances() {
instances = StartInstances()
WaitForInstancesToBecomeRunning(instances)
WaitForInstancesToAcceptSSHConnection(instances).
return (instances)
}
That's nice. What is happening is very clear, and the order of program actions is very clear. No white noise to distract you from understanding the code and the flow. I'd really like to end up with code that looks like that.
But in reality, I can't have a synchronous solution .. each of those functions can run for a long time, and each needs to do things like: update the ui, monitor for time-outs being exceeded, and retry operations periodically until success or time-out. In short, each of these needs to be happening in the background so the foreground UI thread can continue on.
But if I use solutions like BackgroundWorker, it seems like I don't end up with nice easy to follow program logic like the above. Instead I might start a background worker from my UI thread to perform the first function, and then my ui thread goes back to the UI while the worker thread runs. When it finishes, its "done" event handler might start the next Background Worker. WHen it finishes, its "done" event handler might start the last BackgroundWorker, and so on. Meaning you have to "follow the trail" of the Done Event handlers in order to understand the overall program flow.
There has to be a better way that a) lets my UI thread be responsive, b) let's my async operations be able to update the ui and most importantly c) be able to express my program as series of consecutive steps (as I've shown above) so that someone can understand the resultant code
Any and all input would be greatly appreciated!
Michael

My generalized question is this: how do you write asynchronous code that is still clear and easy to follow, like a synchronous solution would be?
You wait for C# 5. It won't be long now. async/await rocks. You've really described the feature in the above sentence... See the Visual Studio async homepage for tutorials, the language spec, downloads etc.
At the moment, there really isn't a terribly clean way - which is why the feature was required in the first place. Asynchronous code very naturally becomes a mess, especially when you consider error handling etc.
Your code would be expressed as:
async Task<List<Instance>> StartNewInstances() {
List<Instance> instances = await StartInstancesAsync();
await instances.ForEachAsync(x => await instance.WaitUntilRunningAsync());
await instances.ForEachAsync(x => await instance.WaitToAcceptSSHConnectionAsync());
return instances;
}
That's assuming a little bit of extra work, such as an extension method on IEnumerable<T> with the form
public static Task ForEachAsync<T>(this IEnumerable<T> source,
Func<T, Task> taskStarter)
{
// Stuff. It's not terribly tricky :(
}

On the off chance that you can't wait for 5 as Jon rightly suggests, I'd suggest that you look at the Task Parallel Library (part of .NET 4). It provides a lot of the plumbing around the "Do this asynchronously, and when it finishes do that" paradigm that you describe in the question. It also has solid support for error handling in the asynchronous tasks themselves.

Async/await is really the best way to go. However, if you don't want to do wait, you can try Continuation-passing-style, or CPS. To do this, you pass a delegate into the async method, which is called when processing is complete. In my opinion, this is cleaner than having all of the extra events.
That will change this method signature
Foo GetFoo(Bar bar)
{
return new Foo(bar);
}
To
void GetFooAsync(Bar bar, Action<Foo> callback)
{
Foo foo = new Foo(bar);
callback(foo);
}
Then to use it, you would have
Bar bar = new Bar();
GetFooAsync(bar, GetFooAsyncCallback);
....
public void GetFooAsyncCallback(Foo foo)
{
//work with foo
}
This gets a little tricky when GetFoo could throw an exception. The method I prefer is to chage the signature of GetFooAsync.
void GetFooAsync(Bar bar, Action<Func<Foo>> callback)
{
Foo foo;
try
{
foo = new Foo(bar);
}
catch(Exception ex)
{
callback(() => {throw ex;});
return;
}
callback(() => foo);
}
Your callback method will look like this
public void GetFooAsyncCallback(Func<Foo> getFoo)
{
try
{
Foo foo = getFoo();
//work with foo
}
catch(Exception ex)
{
//handle exception
}
}
Other methods involve giving the callback two parameters, the actual result and an exception.
void GetFooAsync(Bar bar, Action<Foo, Exception> callback);
This relies on the callback checking for an exception, which could allow it to be ignored. Other methods have two call backs, one for success, and one for failure.
void GetFooAsync(Bar bar, Action<Foo> callback, Action<Exception> error);
To me this makes the flow more complicated, and still allows the Exception to be ignored.
However, giving the callback a method that must be called to get the result forces the callback to deal with the Exception.

When it finishes, its "done" event handler might start the next Background Worker.
This is something that I've been struggling with for a while. Basically waiting for a process to finish without locking the UI.
Instead of using a backgroundWorker to start a backgroundWorker however, you can just do all the tasks in one backgroundWorker. Inside the backgroundWorker.DoWork function, it runs synchronously on that thread. So you can have one DoWork function that processes all 3 items.
Then you have to wait on just the one BackgroundWorker.Completed and have "cleaner" code.
So you can end up with
BackgroundWorker_DoWork
returnValue = LongFunction1
returnValue2 = LongFunction2(returnValue)
LongFunction3
BackgroundWorker_ProgressReported
Common Update UI code for any of the 3 LongFunctions
BackgroundWorker_Completed
Notify user long process is done

In some scenario (will explain later), you can wrap the async calls to a method like the following pseudo code:
byte[] ReadTheFile() {
var buf = new byte[1000000];
var signal = new AutoResetEvent(false);
proxy.BeginReadAsync(..., data => {
data.FillBuffer(buf);
signal.Set();
});
signal.WaitOne();
return buf;
}
For the above code to work, the call back needs to be invoked from a different thread. So this depends on what you are working with. From my experience, at least Silverlight web service calls are handled in UI thread, which means the above pattern cannot be used - if the UI thread is blocked, the previous begin call even cannot be carried out. If you are working with this kind of frameworks, another way to handle multiple async calls is to move your higher level logic to a background thread and use UI thread for communication. However, this approach is a little bit over killing in most cases because it requires some boilerplate code to start and stop background thread.

Asynchronous Calls Newbie Question

I am working on a small project where I need to make two asynchronous calls right after another.
My code looks something like this:
AsynchronousCall1();
AsynchronousCall2();
The problem I'm having is that both calls take anywhere from one to two seconds to execute and I never know which one will finish last. What I'm looking for is a way to determine who finishes last. If Call1() finishes last, I do one thing. If Call2() finishes last, I do another thing.

This is a simple example of using a lock to ensure that only one thread can enter a piece of code. But it's a general example, which may or may not be best for your application. Add some details to your question to help us find what you're looking for.
void AsynchronousCall1()
{
// do some work
Done("1");
}
void AsynchronousCall2()
{
// do some work
Done("2");
}
readonly object _exclusiveAccess = new object();
volatile bool _alreadyDone = false;
void Done(string who)
{
lock (_exclusiveAccess)
{
if (_alreadyDone)
return;
_alreadyDone = true;
Console.WriteLine(who + " was here first");
}
}

I believe there is a method that is a member of the Thread class to check on a specific thread and determine its status. The other option would be to use a BackgroundWorker instead, as that would allow you to spell out what happens when that thread is finished, by creating seperate methods.
The "thread-unsafe" option would be to use a class variable, and at the end of each thread if it isn't locked / already have the other thread's value, don't set it. Otherwise set it.
Then when in your main method, after the call to wait for all threads to finish, test the class variable.
That will give you your answer as to which thread finished first.

You can do this with two ManualResetEvent objects. The idea is to have the main thread initialize both to unsignaled and then call the asynchronous methods. The main thread then does a WaitAny on both objects. When AsynchronousCall1 completes, it signals one of the objects. When AsynchronousCall2 completes, it signals the other. Here's code:
ManualResetEvent Event1 = new ManualResetEvent(false);
ManualResetEvent Event2 = new ManualResetEvent(false);
void SomeMethod()
{
WaitHandle[] handles = {Event1, Event2};
AsynchronousCall1();
AsynchronousCall2();
int index = WaitHandle.WaitAny(handles);
// if index == 0, then Event1 was signaled.
// if index == 1, then Event2 was signaled.
}
void AsyncProc1()
{
// does its thing and then
Event1.Signal();
}
void AsyncProc2()
{
// does its thing and then
Event2.Signal();
}
There are a couple of caveats here. If both asynchronous methods finish before the call to WaitAny, it will be impossible to say which completed first. Also, if both methods complete very close to one another (i.e. call 1 completes, then call 2 completes before the main thread's wait is released), it's impossible to say which one finished first.

You may want to check out the Blackboard design pattern: http://chat.carleton.ca/~narthorn/project/patterns/BlackboardPattern-display.html. That pattern sets up a common data store and then lets agents (who know nothing about one another -- in this case, your async calls) report their results in that common location. Your blackboard's 'supervisor' would then be aware of which call finished first and could direct your program accordingly.