We Keep Coding

Task.Wait for async method passes during application startup while it causes deadlock in WPF button handler

The behavior of Task.Wait() is unexpectedly different depending on the "environment" where invoked.
Calling Task.Wait() during application startup with below async method TestAsync passes (doesn't cause a deadlock) while the same code blocks when called from within a WPF Button handler.
Steps to reproduce:
In Visual Studio, using the wizard, create a vanilla WPF .NET framework application (e.g. named WpfApp).
In the App.xaml.cs file of the app file paste below Main method and TestAsync method.
In the project properties set Startup object to WpfApp.App.
In the properties of App.xaml switch Build Action from ApplicationDefinition to Page.
public partial class App : Application
{
[STAThread]
public static int Main(string[] args)
{
Task<DateTime> task = App.TestAsync();
task.Wait();
App app = new App();
app.InitializeComponent();
return app.Run();
}
internal static async Task<DateTime> TestAsync()
{
DateTime completed = await Task.Run<DateTime>(() => {
System.Threading.Thread.Sleep(3000);
return DateTime.Now;
});
System.Diagnostics.Debug.WriteLine(completed);
return completed;
}
}
Observe that the application starts properly (after 3sec delay) and that the "completed" DateTime is written to debug output.
Next create a Button in MainWindow.xaml with Click handler Button_Click in MainWindow.xaml.cs
public partial class MainWindow : Window
{
...
private void Button_Click(object sender, RoutedEventArgs e)
{
Task<DateTime> task = App.TestAsync();
task.Wait();
}
}
Observe that after clicking the Button, the application is deadlocked.
Why can't it pass in both cases?
Is there a way to change invocation (e.g. using ConfigureAwait at the correct task or somehow setting SynchronizationContext or whatever) so that it behaves identical in both invocations, but still synchronously waits for completion?
Update on limitations of the solution.
The async method like TestAsync comes from a library that cannot be changed.
The invocation code of the TestAsync method is nested within a callstack that cannot be changed either, and the code outside the callstck makes use of the returned value of the async method.
Ultimately the solution code has to convert the async method to run synchronous by not changing the method nor the caller.
This works well within UT code (NUnit) and during application startup, but no more within a handler of WPF.
Why?

There are a couple of different ways that you can handle this situation, but ultimately the reason there is a deadlock in one situation and not the other is that when called in the Main method SynchronizationContext.Current is null, so there isn't a main UI context to capture and all async callbacks are handled on thread pool threads. When called from the button, there is a synchronization context which is captured automatically, so all async callbacks in that situation are handled on the main UI thread which is causing the deadlock. In general the only way you won't get that deadlock is by forcing the async code to not capture the synchronization context, or use async all the way up and don't synchronously wait from the main UI context.
you can ConfigureAwait(false) inside of your TestAsync method so that it doesn't capture the synchronization context and try to continue on the main UI thread (this is ultimately what is causing your deadlock because you are calling task.Wait() on the UI thread which is blocking the UI thread, and you have System.Diagnostics.Debug.WriteLine(completed); that is trying to be scheduled back onto the UI thread because await automatically captures the synchronization context)
DateTime completed = await Task.Run<DateTime>(() => {
System.Threading.Thread.Sleep(3000);
return DateTime.Now;
}).ConfigureAwait(false);
You can start the async task on a background thread so that there isn't a synchronization context to capture.
private void Button_Click(object sender, RoutedEventArgs e)
{
var task = Task.Run(() => App.TestAsync());
var dateTime = task.Result;
}
you can use async up the whole stack
private async void Button_Click(object sender, RoutedEventArgs e)
{
Task<DateTime> task = App.TestAsync();
var dateTime = await task;
}
Given how you are using it, if you don't have to wait until the task is done, you can just let it go and it will finish eventually, but you lose the context to handle any exceptions
private void Button_Click(object sender, RoutedEventArgs e)
{
//assigning to a variable indicates to the compiler that you
//know the application will continue on without checking if
//the task is finished. If you aren't using the variable, you
//can use the throw away special character _
_ = App.TestAsync();
}
These options are not in any particular order, and actually, best practice would probably be #3. async void is allowed specifically for cases like this where you want to handle a callback event asynchronously.

From what I understand, in .NET many of the front ends have a single UI thread, and therefore must be written async all the way through. Other threads are reserved and utilized for things like rendering.
For WPF, this is why use of the Dispatcher and how you queue up work items is important, as this is your way to interact with the one thread you have at your disposal. More reading on it here
Ditch the .Result as this will block, rewrite the method as async, and call it from within the Dispatch.Invoke() and it should run as intended

Why can't it pass in both cases?
The difference is the presence of a SynchronizationContext. All threads start out without a SynchronizationContext. UI applications have a special UI thread(s) and at some point they need to create a SynchronizationContext and install it on that thread(s). Exactly when this happens isn't documented (or consistent), but it has to be installed at the point the UI main loop starts.
In this case, WPF will install it (at the latest) within the call to Application.Run. All user invocations from the UI framework (e.g., event handlers) happen within this context.
The blocking code deadlocks with the context because this is the classic deadlock situation, which requires three components:
A context that only allows one thread at a time.
An asynchronous method that captures that context.
A method also running in that context that blocks waiting for that asynchronous method.
Before the WPF code installed the context, condition (1) wasn't met, and that's why it didn't deadlock.
Is there a way to change invocation (e.g. using ConfigureAwait at the correct task or somehow setting SynchronizationContext or whatever) so that it behaves identical in both invocations, but still synchronously waits for completion?
We-ell...
This is a rephrasing of "how do I block on asynchronous code", and there's no good answer for that. The best answer is to not block on asynchronous code at all; i.e., use async all the way. Especially since this is GUI code, I'd say for the sake of UX you really want to avoid blocking. Since you're on WPF, you may find a technique like asynchronous MVVM data binding useful.
That said, there are a few hacks you can use if you must. Using ConfigureAwait is one possible solution, but not one I recommend; you'd have to apply it to all awaits within the transitive closure of all methods being blocked on (Blocking Hack). Or you can shunt the work to the thread pool (Task.Run) and block on that (Thread Pool Hack). Or you can remove the SynchronizationContext - unless the code being blocked on manipulates UI elements or bound data. Or there are even more dangerous hacks that I really can't recommend at all (Nested Message Loop Hack).
But even after putting in all the work for a hack, you'll still end up blocking the UI. The hacks are hard precisely because they're not recommended. It's quite a bit of work to give your users a worse experience. The far, far better solution (for your users and future code maintainers) is to go async all the way.

Why the default SynchronizationContext is not captured in a Console App?

I'm trying to learn more about the SynchronizationContext, so I made this simple console application:
private static void Main()
{
var sc = new SynchronizationContext();
SynchronizationContext.SetSynchronizationContext(sc);
DoSomething().Wait();
}
private static async Task DoSomething()
{
Console.WriteLine(SynchronizationContext.Current != null); // true
await Task.Delay(3000);
Console.WriteLine(SynchronizationContext.Current != null); // false! why ?
}
If I understand correctly, the await operator captures the current SynchronizationContext then posts the rest of the async method to it.
However, in my application the SynchronizationContext.Current is null after the await. Why is that ?
EDIT:
Even when I use my own SynchronizationContext it is not captured, although its Post function is called. Here is my SC:
public class MySC : SynchronizationContext
{
public override void Post(SendOrPostCallback d, object state)
{
base.Post(d, state);
Console.WriteLine("Posted");
}
}
And this is how I use it:
var sc = new MySC();
SynchronizationContext.SetSynchronizationContext(sc);
Thanks!

The word "capture" is too opaque, it sounds too much like that is something that the framework is supposed to. Misleading, since it normally does in a program that uses one of the default SynchronizationContext implementations. Like the one you get in a Winforms app. But when you write your own then the framework no longer helps and it becomes your job to do it.
The async/await plumbing gives the context an opportunity to run the continuation (the code after the await) on a specific thread. That sounds like a trivial thing to do, since you've done it so often before, but it is in fact quite difficult. It is not possible to arbitrarily interrupt the code that this thread is executing, that would cause horrible re-entrancy bugs. The thread has to help, it needs to solve the standard producer-consumer problem. Takes a thread-safe queue and a loop that empties that queue, handling invoke requests. The job of the overridden Post and Send methods is to add requests to the queue, the job of the thread is to use a loop to empty it and execute the requests.
The main thread of a Winforms, WPF or UWP app has such a loop, it is executed by Application.Run(). With a corresponding SynchronizationContext that knows how to feed it with invoke requests, respectively WindowsFormsSynchronizationContext, DispatcherSynchronizationContext and WinRTSynchronizationContext. ASP.NET can do it too, uses AspNetSynchronizationContext. All provided by the framework and automagically installed by the class library plumbing. They capture the sync context in their constructor and use Begin/Invoke in their Post and Send methods.
When you write your own SynchronizationContext then you must now take care of these details. In your snippet you did not override Post and Send but inherited the base methods. They know nothing and can only execute the request on an arbitrary threadpool thread. So SynchronizationContext.Current is now null on that thread, a threadpool thread does not know where the request came from.
Creating your own isn't that difficult, ConcurrentQueue and delegates help a lot of cut down on the code. Lots of programmers have done so, this library is often quoted. But there is a severe price to pay, that dispatcher loop fundamentally alters the way a console mode app behaves. It blocks the thread until the loop ends. Just like Application.Run() does.
You need a very different programming style, the kind that you'd be familiar with from a GUI app. Code cannot take too long since it gums up the dispatcher loop, preventing invoke requests from getting dispatched. In a GUI app pretty noticeable by the UI becoming unresponsive, in your sample code you'll notice that your method is slow to complete since the continuation can't run for a while. You need a worker thread to spin-off slow code, there is no free lunch.
Worthwhile to note why this stuff exists. GUI apps have a severe problem, their class libraries are never thread-safe and can't be made safe by using lock either. The only way to use them correctly is to make all the calls from the same thread. InvalidOperationException when you don't. Their dispatcher loop help you do this, powering Begin/Invoke and async/await. A console does not have this problem, any thread can write something to the console and lock can help to prevent their output from getting intermingled. So a console app shouldn't need a custom SynchronizationContext. YMMV.

By default, all threads in console applications and Windows Services only have the default SynchronizationContext.
Kindly refer to the MSDN article Parallel Computing - It's All About the SynchronizationContext. This has detailed information regarding SynchronizationContexts in various types of applications.

To elaborate on what was already pointed out.
The SynchronizationContext class that you use in the first code snippet is the default implementation, which doesn't do anything.
In the second code snippet, you create your own MySC context. But you are missing the bit that would actually make it work:
public override void Post(SendOrPostCallback d, object state)
{
base.Post(state2 => {
// here we make the continuation run on the original context
SetSynchronizationContext(this);
d(state2);
}, state);
Console.WriteLine("Posted");
}

Implementing your own SynchronizationContext is doable, but not trivial. It's much easier to use an existing implementation, like the AsyncContext class from the Nito.AsyncEx.Context package. You can use it like this:
using System;
using System.Threading;
using System.Threading.Tasks;
using Nito.AsyncEx;
public static class Program
{
static void Main()
{
AsyncContext.Run(async () =>
{
await DoSomethingAsync();
});
}
static async Task DoSomethingAsync()
{
Console.WriteLine(SynchronizationContext.Current != null); // True
await Task.Delay(3000);
Console.WriteLine(SynchronizationContext.Current != null); // True
}
}
Try it on Fiddle.
The AsyncContext.Run is a blocking method. It will complete when the supplied asynchronous delegate Func<Task> action completes. All asynchronous continuations are going to run on the console application's main thread, provided that there is no Task.Run or ConfigureAwait(false) that would force your code to exit the context.
The consequences of using a single-threaded SynchronizationContext in a console application are that:
You'll no longer have to worry about thread-safety, since all your code will be funneled to a single thread.
Your code becomes susceptible to deadlocks. Any .Wait(), .Result, .GetAwaiter().GetResult() etc inside your code is very likely to cause your application to freeze, in which case you'll have to kill the process manually from the Windows Task Manager.

async Task - What actually happens on the CPU?

I've been reading about Tasks after asking this question and seeing that I completely misunderstood the concept. Answers such as the top answers here and here explain the idea, but I still don't get it.
So I've made this a very specific question: What actually happens on the CPU when a Task is executed?
This is what I've understood after some reading: A Task will share CPU time with the caller (and let's assume the caller is the "UI") so that if it's CPU-intensive - it will slow down the UI. If the Task is not CPU-intensive - it will be running "in the background". Seems clear enough …… until tested. The following code should allow the user to click on the button, and then alternately show "Shown" and "Button". But in reality: the Form is completely busy (-no user input possible) until the "Shown"s are all shown.
public Form1()
{
InitializeComponent();
Shown += Form1_Shown;
}
private async void Form1_Shown(object sender, EventArgs e)
{
await Doit("Shown");
}
private async Task Doit(string s)
{
WebClient client = new WebClient();
for (int i = 0; i < 10; i++)
{
client.DownloadData(uri);//This is here in order to delay the Text writing without much CPU use.
textBox1.Text += s + "\r\n";
this.Update();//textBox1.
}
}
private async void button1_Click(object sender, EventArgs e)
{
await Doit("Button");
}
Can someone please tell me what is actually happening on the CPU when a Task is executed (e.g. "When the CPU is not used by the UI, the Task uses it, except for when… etc.")?

The key to understanding this is that there are two kinds of tasks - one that executes code (what I call Delegate Tasks), and one that represents a future event (what I call Promise Tasks). Those two tasks are completely different, even though they're both represented by an instance of Task in .NET. I have some pretty pictures on my blog that may help understand how these types of task are different.
Delegate Tasks are the ones created by Task.Run and friends. They execute code on the thread pool (or possibly another TaskScheduler if you're using a TaskFactory). Most of the "task parallel library" documentation deals with Delegate Tasks. These are used to spread CPU-bound algorithms across multiple CPUs, or to push CPU-bound work off a UI thread.
Promise Tasks are the ones created by TaskCompletionSource<T> and friends (including async). These are the ones used for asynchronous programming, and are a natural fit for I/O-bound code.
Note that your example code will cause a compiler warning to the effect that your "asynchronous" method Doit is not actually asynchronous but is instead synchronous. So as it stands right now, it will synchronously call DownloadData, blocking the UI thread until the download completes, and then it will update the text box and finally return an already-completed task.
To make it asynchronous, you have to use await:
private async Task Doit(string s)
{
WebClient client = new WebClient();
for (int i = 0; i < 10; i++)
{
await client.DownloadDataTaskAsync(uri);
textBox1.Text += s + "\r\n";
this.Update();//textBox1.
}
}
Now it's returning an incomplete task when it hits the await, which allows the UI thread to return to its message processing loop. When the download completes, the remainder of this method will be queued to the UI thread as a message, and it will resume executing that method when it gets around to it. When the Doit method completes, then the task it returned earlier will complete.
So, tasks returned by async methods logically represent that method. The task itself is a Promise Task, not a Delegate Task, and does not actually "execute". The method is split into multiple parts (at each await point) and executes in chunks, but the task itself does not execute anywhere.
For further reading, I have a blog post on how async and await actually work (and how they schedule the chunks of the method), and another blog post on why asynchronous I/O tasks do not need to block threads.

As per your linked answers, Tasks and Threads are totally different concepts, and you are also getting confused with async / await
A Task is just a representation of some work to be done. It says nothing about HOW that work should be done.
A Thread is a representation of some work that is running on the CPU, but is sharing the CPU time with other threads that it can know nothing about.
You can run a Task on a Thread using Task.Run(). Your Task will run asynchronously and independently of any other code providing a threadpool thread is available.
You can also run a Task asynchronously on the SAME thread using async / await. Anytime the thread hits an await, it can save the current stack state, then travel back up the stack and carry on with other work until the awaited task has finished. Your Doit() code never awaits anything, so will run synchronously on your GUI thread until complete.

Tasks use the ThreadPool you can read extensively about what it is and how it works here
But in a nutshell, when a task is executed, the Task Scheduler looks in the ThreadPool to see if there is a thread available to run the action of the task. If not, it's going to be queued until one becomes available.
A ThreadPool is just a collection of already-instantiated threads made available so that multithreaded code can safely use concurrent programming without overwhelming the CPU with context-switching all the time.
Now, the problem with your code is that even though you return an object of type Task, you are not running anything concurrently - No separate thread is ever started!
In order to do that, you have two options, either you start yourDoit method as a Task, with
Option1
Task.Run(() => DoIt(s));
This will run the whole DoIt method on another thread from the Thread Pool, but it will lead to more problems, because in this method, you're trying to access UI-controls. therefore, you will need either to marshal those calls to the UI thread, or re-think your code so that the UI access is done directly on the UI thread after the asynchronous tasks completes.
Option 2 (preferred, if you can)
You use .net APIs which are already asynchronous, such as client.DownloadDataTaskAsync(); instead of client.DownloadData();
now, in your case, the problem is that you will need to have 10 calls, which are going to return 10 different objects of type Task<byte[]> and you want to await on the completion of all of them, not just one.
In order to do this, you will need to create a List<Task<byte[]>> returnedTasks and you will add to it all returned value from DownloadDataTaskAsync(). then, once this is done, you can use the following return value for your DoIt method.
return Task.WhenAll(returnedTasks);

When I cannot use ConfigureAwait(false)?

According to best practices it is recommended to use .ConfigureAwait(false) with async/await keywords if you can:
await Task.Run(RunSomethingAsync).ConfigureAwait(false);
Can you please give me an example of a situation when I cannot use .ConfigureAwait(false)?

You "cannot" use ConfigureAwait(false) when you actually care about the synchronization context you're in. For example, imagine the following in a GUI application:
public async void SomeButtonClick(object sender, EventArgs e)
{
var result = await SomeAsyncOperation().ConfigureAwait(false);
textBox.Text = result;
}
When you return from ConfigureAwait, you wont be back on the UI thread. This will cause an InvalidOperationException.

From the source: Asynchronous .NET Client Libraries for Your HTTP API and Awareness of async/await's Bad Effects:
When you are awaiting on a method with await keyword, compiler
generates bunch of code in behalf of you. One of the purposes of this
action is to handle synchronization with the UI (or main) thread. The key
component of this feature is the SynchronizationContext.Current which
gets the synchronization context for the current thread.
SynchronizationContext.Current is populated depending on the
environment you are in. The GetAwaiter method of Task looks up for
SynchronizationContext.Current. If current synchronization context is
not null, the continuation that gets passed to that awaiter will get
posted back to that synchronization context.
When consuming a method, which uses the new asynchronous language
features, in a blocking fashion, you will end up with a deadlock if
you have an available SynchronizationContext. When you are consuming
such methods in a blocking fashion (waiting on the Task with Wait
method or taking the result directly from the Result property of the
Task), you will block the main thread at the same time. When
eventually the Task completes inside that method in the threadpool, it
is going to invoke the continuation to post back to the main thread
because SynchronizationContext.Current is available and captured. But
there is a problem here: the UI thread is blocked and you have a
deadlock!

How does running several tasks asynchronously on UI thread using async/await work?

I've read (and used) async/await quite a lot for some time now but I still have one question I can't get an answer to. Say I have this code.
private async void workAsyncBtn_Click(object sender, EventArgs e)
{
var myTask = _asyncAwaitExcamples.DoHeavyWorkAsync(5);
await myTask;
statusTextBox.Text += "\r\n DoHeavyWorkAsync message";
}
It's called from the UI thread and returned to the UI Thread. Therefor I am able to do UI-specific things in this method and after the await myTask. If I had used .ConfigureAwait(false) I would get a thread exception when doing statusTextBox.Text += "\r\n DoHeavyWorkAsync message"; since I would have telled myTask it's ok to take any available thread from the thread pool.
My question. As I understand it I never leave the UI thread in this case, still it's run asynchronously, the UI is still responsive and I can start several Tasks at the same time and therefor speed up my application. How can this work if we only use one thread?
Thanks!
EDIT for Sievajet
private async void workAsyncBtn_Click(object sender, EventArgs e)
{
await DoAsync();
}
private async Task DoAsync()
{
await Task.Delay(200);
statusTextBox.Text += "Call to form";
await Task.Delay(200);
}

As I understand it I never leave the UI thread in this case, still
it's run asynchronously, the UI is still responsive and I can start
several Tasks at the same time and therefor speed up my application.
How can this work if we only use one thread?
First, i'd recommend reading Stephan Clearys blog post - There is no thread.
In order to understand how its possible to run multiple units of work altogether, we need to grasp one important fact: async IO bound operations have (almost) nothing to do with threads.
How is that possible? well, if we drill deep down all the way to the operating system, we'll see that the calls to the device drivers - those which are in charge of doing operations such as network calls and writing to disk, were all implemented as naturally asynchronous, they don't occupy a thread while doing their work. That way, while the device driver is doing its thing, there need not be a thread. only once the device driver completes its execution, it will signal the operating system that it's done via an IOCP (I/O completion port), which will then execute the rest of the method call (this is done in .NET via the threadpool, which has dedicated IOCP threads).
Stephans blog post demonstrates this nicely:
Once the OS executes the DPC (Deferred Procedure Call) and queue the IRP (I/O Request Packet), it's work is essentially done until the device driver signals it back with the I'm done messages, which causes a whole chain of operations (described in the blog post) to execute, which eventually will end up with invoking your code.
Another thing to note is that .NET does some "magic" for us behind the scenes when using async-await pattern. There is a thing called "Synchronization Context" (you can find a rather lengthy explanation here). This sync context is whats in-charge of invoking the continuation (code after the first await) on the UI thread back again (in places where such context exists).
Edit:
It should be noted that the magic with the synchronization context happens for CPU bound operations as well (and actually for any awaitable object), so when you use a threadpool thread via Task.Run or Task.Factory.StartNew, this will work as well.

The TaskParallelLibrary (TPL) uses a TaskScheduler which can be configured with TaskScheduler.FromCurrentSynchronizationContext to return to the SynchronizationContext like this :
textBox1.Text = "Start";
// The SynchronizationContext is captured here
Factory.StartNew( () => DoSomeAsyncWork() )
.ContinueWith(
() =>
{
// Back on the SynchronizationContext it came from
textBox1.Text = "End";
},TaskScheduler.FromCurrentSynchronizationContext());
When an async method suspends at an await, by default it will capture the current SynchronizationContext and marshall the code after the await back on the SynchronizationContext it came from.
textBox1.Text = "Start";
// The SynchronizationContext is captured here
/* The implementation of DoSomeAsyncWork depends how it runs, this could run on the threadpool pool
or it could be an 'I/O operation' or an 'Network operation'
which doesnt use the threadpool */
await DoSomeAsyncWork();
// Back on the SynchronizationContext it came from
textBox1.Text = "End";
async and await example:
async Task MyMethodAsync()
{
textBox1.Text = "Start";
// The SynchronizationContext is captured here
await Task.Run(() => { DoSomeAsyncWork(); }); // run on the threadPool
// Back on the SynchronizationContext it came from
textBox1.Text = "End";
}

When UI thread calls await it starts the async operation and returns immediately. When the async operation completes, it notifies a thread from the thread pool but the internal implementation of async await dispatches the execution to the UI thread which will continue the execution of the code after the await.
The Dispatch is implemented by means of SynchronizationContext which in turn calls System.Windows.Forms.Control.BeginInvoke.
CLR via C# (4th Edition) (Developer Reference) 4th Edition by Jeffrey Richter page 749
Actually, Jeffrey worked with MS to implement the async/await inspired by his AsyncEnumerator