We Keep Coding

Practical example of async await does not meet expectations

Related to this question: Does await completely blocks the thread?
[...] it will first check to see if the called method completed, and if not will register the continuation and return from that method call. Later, once that method completes, it will re-enter the state-machine in order to complete the method
And to this question also: When is the best place to use Task.Result instead of awaiting Task
await simply means "this workflow cannot progress further until this task is completed, so if it is not complete, find more work to do and come back later"
And finally to this post: https://blog.stephencleary.com/2012/02/async-and-await.html
If “await” sees that the awaitable has not completed, then it acts asynchronously. It tells the awaitable to run the remainder of the method when it completes, and then returns from the async method. Later on, when the awaitable completes, it will execute the remainder of the async method. If you’re awaiting a built-in awaitable (such as a task), then the remainder of the async method will execute on a “context” that was captured before the “await” returned.
So from these posts I get that the await operator does indeed not block, but when I've tried to test it i just cannot get this principle to work the way it states to work. Obviously I'm missing something:
//This will take 10 seconds
[HttpGet("test1")]
public async Task<TimeSpan> test()
{
var t1 = DateTime.Now;
var wait1 = DoAsyncEcho("The first!", 10000);
var wait2 = DoAsyncEcho("The second!", 10000);
_logger.LogInformation(await wait1);
_logger.LogInformation(await wait2);
_logger.LogInformation("DONE!");
var t2 = DateTime.Now;
return t2 - t1;
}
//This will take 10 seconds too
[HttpGet("test2")]
public async Task<TimeSpan> test2()
{
var t1 = DateTime.Now;
var wait1 = DoAsyncEcho("The first!", 10000);
var wait2 = DoAsyncEcho("The second!", 10000);
Thread.Sleep(10000);
_logger.LogInformation(await wait1);
_logger.LogInformation(await wait2);
_logger.LogInformation("DONE!");
var t2 = DateTime.Now;
return t2 - t1;
}
//This will take 20
[HttpGet("test3")]
public async Task<TimeSpan> test3()
{
var t1 = DateTime.Now;
var wait1 = await DoAsyncEcho("The first!", 10000);
var wait2 = await DoAsyncEcho("The second!", 10000);
_logger.LogInformation(wait1);
_logger.LogInformation(wait2);
_logger.LogInformation("DONE!");
var t2 = DateTime.Now;
return t2 - t1;
}
//This will take 30
[HttpGet("test4")]
public async Task<TimeSpan> test4()
{
var t1 = DateTime.Now;
var wait1 = await DoAsyncEcho("The first!", 10000);
var wait2 = await DoAsyncEcho("The second!", 10000);
Thread.Sleep(10000);
_logger.LogInformation(wait1);
_logger.LogInformation(wait2);
_logger.LogInformation("DONE!");
var t2 = DateTime.Now;
return t2 - t1;
}
private Task<string> DoAsyncEcho(string v, int t)
{
return Task<string>.Factory.StartNew(() =>
{
Thread.Sleep(t);
return v;
}
);
}
As I see from the methods test3 and test4, await does indeed wait, it does not enter into a state-machine and does a callback later on because it waits the full 10 seconds of the first DoAsyncEcho and then another 10s on the second call. On the methods test1 and test2 execution time lasts for 10s as code does not waits for the return of the DoAsyncEcho but only it awaits for the result later. Particulary test2 method sleeps the 3 calls of 10 seconds in parallel so after all it's just a 10s run.
What do I'm missing here?

I think the best way to demonstrate this is through a simple Windows Forms app.
Create a default Windows Forms app and drop 3 buttons onto it (called button1, button2 and button3.
Then add the following code:
async void button1_Click(object sender, EventArgs e)
{
this.Text = "[button1_Click] About to await slowMethodAsync()";
int result = await slowMethodAsync();
this.Text = "[button1_Click] slowMethodAsync() returned " + result;
}
void button2_Click(object sender, EventArgs e)
{
this.Text = "[button2_Click] About to start task to call slowMethod()";
int result = 0;
Task.Run(() =>
{
result = slowMethod();
}).ContinueWith(_ =>
{
this.Invoke(new Action(() =>
{
this.Text = "[button2_Click] slowMethod() returned " + result;
}));
});
}
void button3_Click(object sender, EventArgs e)
{
this.Text = "[button3_Click] About to call slowMethod()";
int result = slowMethod();
this.Text = "[button3_Click] slowMethod() returned " + result;
}
static async Task<int> slowMethodAsync()
{
await Task.Delay(5000);
return 42;
}
static int slowMethod()
{
Thread.Sleep(5000);
return 42;
}
If you try this code out, you will notice the following:
Pressing button1 will immediately change the title to [button1_Click] About to await Task.Delay(5000), and you can resize the dialog while waiting for 5 seconds, after which the title will change to [button1_Click] Awaited Task.Delay(5000).
The code for handling button2 is very roughly equivalent to the state machine that is generated from the await code for button1. If you press button2, you will see similar effects to pressing button1.
(The actual code for await is in reality quite different, but the underlying mechanism of using a continuation - i.e., ContinueWith() and Invoke(), to continue executing the code after the await on the UI thread illustrates its approach.)
The code for button3 completely blocks during the Thread.Sleep(), and if you press button3 the UI locks up completely for 5 seconds,.
To illustrate what happens with a non-UI example, consider the following console application:
using System;
using System.Threading;
using System.Threading.Tasks;
namespace Demo
{
static class Program
{
static async Task Main()
{
Console.WriteLine("Main thread ID = " + Thread.CurrentThread.ManagedThreadId);
int result = slowMethod();
Console.WriteLine("result = " + result);
Console.WriteLine("After calling slowMethod(), thread ID = " + Thread.CurrentThread.ManagedThreadId);
result = await slowMethodAsync();
Console.WriteLine("result = " + result);
Console.WriteLine("After calling slowMethodAsync(), thread ID = " + Thread.CurrentThread.ManagedThreadId);
}
static async Task<int> slowMethodAsync()
{
await Task.Delay(5000);
return 42;
}
static int slowMethod()
{
Thread.Sleep(5000);
return 42;
}
}
}
If you run that, you will see output similar to the following:
Main thread ID = 1
result = 42
After calling slowMethod(), thread ID = 1
result = 42
After calling slowMethodAsync(), thread ID = 4
Note how the code has resumed on a different thread after the await.
The key thing to realise is that as far as calling code is concerned, y = await X(); does not return until it has a value to return, and the code that runs afterwards may be running on a different thread.
The effect of this in terms of blocking THREADS is that the calling thread is freed up to go off and execute some other code, and another thread is only required when the async method returns.
In many cases, this means that no additional thread is required (for the continuation), and in all cases it means that the original calling thread is not blocked and can be freed up to the thread pool for use for another task.
This is the "non blocking" part of all this.
For a good, detailed explanation of why sometimes no additional thread is needed, read Stephen Cleary's excellent "There is no thread".

It seems that you are confusing two different interpretations of wait and block. The objective of asynchronous code is to block your code, while the thread remains unblocked. If you don't want to block your code, then the solution is easy: don't use await. But if you don't block your code, then you can't use the result of the asynchronous operation, because the asynchronous operation is running concurrently with your code.
What has not happened yet belongs to the future, and the future is unknown. Not only do you not know the result, you don't even know if the operation succeeded or failed. In most cases this is problematic. You need the result of the operation before continuing with processing this result. So you must block your code. And this is why await was invented, to block your code without having to block a thread too.
You need the thread to remain unblocked, so that it continues running the UI message pump that keeps your application responsive. Just because your code is blocked, your _application) need not to be blocked also. For ASP.NET applications, you need the thread to remain unblocked so that it can serve other incoming web requests. The fewer threads you block, the more requests you can serve. In this case async/await becomes a booster of scalability.

async & await - How to wait until all Tasks are done?

ok. I made a simple console app to figure out how to make all this work. Once I have the basic outline working, then I'll apply it to the real application.
The idea is that we have a lot of database calls to execute that we know are going to take a long time. We do NOT want to (or have to) wait for one database call to be completed before we make the next. They can all run at the same time.
But, before making all of the calls, we need to perform a "starting" task. And when all of the calls are complete, we need to perform a "finished" task.
Here's where I'm at now:
static void Main(string[] args)
{
Console.WriteLine("starting");
PrintAsync().Wait();
Console.WriteLine("ending"); // Must not fire until all tasks are finished
Console.Read();
}
// Missing an "await", I know. But what do I await for?
static async Task PrintAsync()
{
Task.Run(() => PrintOne());
Task.Run(() => PrintTwo());
}
static void PrintOne()
{
Console.WriteLine("one - start");
Thread.Sleep(3000);
Console.WriteLine("one - finish");
}
static void PrintTwo()
{
Console.WriteLine("two - start");
Thread.Sleep(3000);
Console.WriteLine("two - finish");
}
But no matter what I try, Ending always gets printed too early:
starting
ending
one - start
two - start
one - finish
two - finish
What IS working right is that PrintTwo() starts before PrintOne() is done. But how do I properly wait for PrintAsync() to finish before doing anything else?

you need to await the ending of the inner tasks:
static async Task PrintAsync()
{
await Task.WhenAll(Task.Run(() => PrintOne()), Task.Run(() => PrintTwo()));
}
explanation: async Task denotes an awaitable method. Inside this method you can also await Tasks. If you don't do this then it will simply let the tasks loose which will run on their own. Task.Run returns a Task which can be awaited. If you want both tasks to run in parallel you can use the tasks from the retur values and use them in the awaitable method Task.WhenAll
EDIT: Actually Visual Studio would mark this code with a green curvy line. When hoovering with the mouse over it you get a warning:
CS4014
This should explain why "ending" is printed before the tasks have finished
EDIT 2:
If you have a collection of parameters that you want to iterate and call an async method to pass the parameter in, you can also do it with a select statement in 1 line:
static async Task DatabaseCallsAsync()
{
// List of input parameters
List<int> inputParameters = new List<int> {1,2,3,4,5};
await Task.WhenAll(inputParameters.Select(x => DatabaseCallAsync($"Task {x}")));
}
static async Task DatabaseCallAsync(string taskName)
{
Console.WriteLine($"{taskName}: start");
await Task.Delay(3000);
Console.WriteLine($"{taskName}: finish");
}
The last part is similar to a previous answer

OP here. I'm going to leave the answer by Mong Zhu marked as correct, as it lead me to the solution. But I also want to share the final result here, which includes excellent feedback in the comments from juharr. Here's what I came up with:
class Program
{
static void Main(string[] args)
{
Console.WriteLine("starting");
DatabaseCallsAsync().Wait();
Console.WriteLine("ending"); // Must not fire until all database calls are complete.
Console.Read();
}
static async Task DatabaseCallsAsync()
{
// This is one way to do it...
var tasks = new List<Task>();
for (int i = 0; i < 3; i++)
{
tasks.Add(DatabaseCallAsync($"Task {i}"));
}
await Task.WhenAll(tasks.ToArray());
// This is another. Same result...
List<int> inputParameters = new List<int> { 1, 2, 3, 4, 5 };
await Task.WhenAll(inputParameters.Select(x => DatabaseCallAsync($"Task {x}")));
}
static async Task DatabaseCallAsync(string taskName)
{
Console.WriteLine($"{taskName}: start");
await Task.Delay(3000);
Console.WriteLine($"{taskName}: finish");
}
}
Here's the result:
starting
Task 0: start
Task 1: start
Task 2: start
Task 2: finish
Task 0: finish
Task 1: finish
ending

What task is being returned and why is this task status RanToCompletion

I am rather new to task based programming and trying to determine how to return a task and verify that it has been started. The code that I got to work was not what I was expecting. The console application is as follows:
public static void Main(string[] args)
{
var mySimple = new Simple();
var cts = new CancellationTokenSource();
var task = mySimple.RunSomethingAsync(cts.Token);
while (task.Status != TaskStatus.RanToCompletion)
{
Console.WriteLine("Starting...");
Thread.Sleep(100);
}
Console.WriteLine("It is started");
Console.ReadKey();
cts.Cancel();
}
public class Simple
{
public async void RunSomething(CancellationToken token)
{
var count = 0;
while (true)
{
if (token.IsCancellationRequested)
{
break;
}
Console.WriteLine(count++);
await Task.Delay(TimeSpan.FromMilliseconds(1000), token).ContinueWith(task => { });
}
}
public Task RunSomethingAsync(CancellationToken token)
{
return Task.Run(() => this.RunSomething(token));
}
}
The output is:
Starting...
0
It is started
1
2
3
4
Why is the task that is being returned have a status as TaskStatus.RanToCompletion compared to TaskStatus.Running as we see that the while loop is still executing? Am I checking the status of the task of putting the RunSomething task on the threadpool rather than the RunSomething task itself?

RunSomething is an async void method, meaning it exposes no means of the caller ever determining when it finishes, they can only ever start the operation and then have no idea what happens next. You then wrap a call to it inside of Task.Run, this is schedluing a thread pool thread to start RunSomething. It will then complete as soon as it has finished starting that Task.
If RunSomething actually returned a Task, then the caller would be able to determine when it actually finished, and if you waited on it it wouldn't actually indicate that it was done until that asynchronous operation was actually finished (there would be no reason to use Task.Run to start it in another thead, you'd be better off just calling it directly and not wasting the effort of moving that to a thread pool thread).

Never use async void (https://msdn.microsoft.com/en-us/magazine/jj991977.aspx)
instead you should use async Task
If you need to call an async method from a non-async (such as from a static void main) you should do something like this:
mySimple.RunSomethingAsync(cts.Token).GetAwaiter().GetResult();
That will effectively make the method a synchronous call.

You can use async void, but only for events.

Program exits upon calling await

I have a while-loop that should repeat the program until a certain condition is met. Inside this loop I call an async function, which prints out a message for me. Here is the (cut-short) code:
private void InitializeMessageSystem ( )
{
do
{
// Do stuff
await printMessage ("Hello World!");
Console.ReadKey();
} while (condition != true)
}
And here the function PrintMessage():
private static async Task PrintMessage (string message, int spd = 1)
{
int delay = 50 / spd;
string[] words = message.Split(' ');
int index = 1;
for (int word = 0; word < words.Length; word++)
{
char[] current = words[word].ToCharArray();
if (index + current.Length > Console.WindowWidth)
{
Console.WriteLine();
index = 1;
}
for (int c = 0; c < current.Length; c++)
{
Console.Write(current[c]);
await Task.Delay(delay);
}
Console.Write(" ");
}
}
Edit: Here's the call from the main function:
static void Main (string[] args)
{
InitializeMessageSystem();
Console.ReadKey();
}
Question
Why does my program exit, when I press a key while the function is not yet completed? I thought the program would wait for the Console.ReadKey() until the function PrintMessage() is completed?

Your problem is that await returns the control flow of the program to the caller of the function. Normally execution is continued at that point when the asynchronous task you await finishes.
So control is returned to your main function as you wait for printMessage and main now waits for a key input. As you hit the key main returns to the OS and your process (including all asynchronous tasks) terminates.
Change your InitializeMessageSystem to
private async Task InitializeMessageSystem ( )
and change the code in main to
InitializeMessageSystem().Wait();
to wait until InitializeMessageSystem finishes completely before waiting for the key.

The below code executes without any errors or warnings. But when you execute the code the program exits silently. What you might be expecting is the program waits for the task to complete, asks the user to press any key and exit. What actually happens is after the await statement is executed the control goes back to the invoking step. In our case after the step await task; is executed, before the task completes the control goes back to the invoking step SomeTask(); and the program exits.
class Program
{
static void Main(string[] args)
{
SomeTask();
}
public static async void SomeTask()
{
Task task = Task.Run(() =>
{
System.Threading.Thread.Sleep(20000);
Console.WriteLine("Task Completed!");
});
await task;
Console.WriteLine("Press any key to exit");
Console.ReadLine();
}
}
To fix this, add await to the SomeTask(); call so that the program waits for async SomeTask() to complete. You should also change the return type
of SomeTask() from void to Task.
class Program
{
static void Main(string[] args)
{
await SomeTask();
}
public static async Task SomeTask()
{
Task task = Task.Run(() =>
{
System.Threading.Thread.Sleep(20000);
Console.WriteLine("Task Completed!");
});
await task;
Console.WriteLine("Press any key to exit");
Console.ReadLine();
}
}

The difference between calling a synchronous function and an async function is that when calling the synchronous function you know that when you reach the statement after the call the function is executed completely. When calling an async function, the function is scheduled as a task at the thread pool to be performed when any of the threads in the pool has time for it.
This gives you time to do other things while one of the threads is performing the task. As soon as you need the result you await for the task to be finished.
This works only if your function is also async. If you don't make your function async your can't use async-await. Making your function async and the clients of your function also async is the only way to truly use async-await. Remember: all async functions should return Task instead of void and Task<TResult> instead of TResult. The only exception is the event handler
private async void Button1_Clicked(object sender, ...)
{
var myTask = SlowMultiplierAsync(4, 3);
// while myTask is running you can do other things
// after a while you need the result:
int taskResult = await myTask;
Process(taskResult);
}
private async Task<int> SlowMultiplierAsync(int x, int y)
{
// let's take a nap before multiplying
// do this by awaiting another async function:
await Task.Delay(TimeSpan.FromSeconds(5));
return x * y;
}
If you don't want (or can) make your function async, you can simulate similar behaviour by starting a task using task.Run:
private void InitializeMessageSystem ( ) {
do
{
// Do stuff
var myTask = task.Run( () => printMessage ("Hello World!"));
myTask.Wait();
Console.ReadKey();
} while (condition != true)
Although this will make sure that your function won't end before the task is finished, your function will be a synchronous one for your clients. For instance it won't make your UI responsive.
Eric lippert here on stackoverflow once explained me the difference between asynchronous concurrent. Link to his answer
Suppose you have to make breakfast. Toast some bread and cook some eggs.
Synchronous: start toasting bread. Wait until toasting finished. Start cooking eggs, wait until eggs are cooked.
Asynchronous but not concurrent: start toasting bread, and while the bread is toasting start cooking eggs. While the eggs are cooking yo can do other things, like making tea. After a while you wait until the eggs are cooked and wait until the bread is toasted. This is typically async-await.
Asynchronous and concurrent: hire a cook to toast the bread, hire a cook to cook the eggs and wait until both cooks are finished. Here the toasting and cooking is done by different threads. It is the most expensive method

Checking if a thread returned to thread pool

How can I check if a thread returned to the thread pool, using VS C# 2015 debugger?
What's problematic in my case is the fact that it cannot be detected by debugging line by line.
async Task foo()
{
int y = 0;
await Task.Delay(5);
// (1) thread 2000 returns to thread pool here...
while (y<5) y++;
}
async Task testAsync()
{
Task task = foo();
// (2) ... and here thread 2000 is back from the thread pool, to run the code below. I want
// to confirm that it was in the thread pool in the meantime, using debugger.
int i = 0;
while (i < 100)
{
Console.WriteLine("Async 1 before: " + i++);
}
await task;
}
In the first line of testAsync running on thread 2000, foo is called. Once it encounters await Task.Delay(5), thread 2000 returns to thread pool (allegedly, I'm trying to confirm this), and the method waits for Task.Delay(5) to complete. In the meantime, the control returns to the caller and the first loop of testAsync is executed on thread 2000 as well.
So between two consecutive lines of code, the thread returned to thread pool and came back from there. How can I confirm this with debugger? Possibly with Threads debugger window?
To clarify a bit more what I'm asking: foo is running on thread 2000. There are two possible scenarios:
When it hits await Task.Delay(5), thread 2000 returns to the thread pool for a very short time, and the control returns to the caller, at line (2), which will execute on thread 2000 taken from the thread pool. If this is true, you can't detect it easily, because Thread 2000 was in the thread pool during time between two consecutive lines of code.
When it hits await Task.Delay(5), thread 2000 doesn't return to thread pool, but immediately executes code in testAsync starting from line (2)
I'd like to verify which one is really happening.

There is a major mistake in your assumption:
When it hits await Task.Delay(5), thread 2000 returns to the thread pool
Since you don't await foo() yet, when thread 2000 hits Task.Delay(5) it just creates a new Task and returns to testAsync() (to int i = 0;). It moves on to the while block, and only then you await task. At this point, if task is not completed yet, and assuming the rest of the code is awaited, thread 2000 will return to the thread pool. Otherwise, if task is already completed, it will synchronously continue from foo() (at while (y<5) y++;).
EDIT:
what if the main method called testAsync?
When synchronous method calls and waits async method, it must block the thread if the async method returns uncompleted Task:
void Main()
{
var task = foo();
task.Wait(); //Will block the thread if foo() is not completed.
}
Note that in the above case the thread is not returning to the thread pool - it is completely suspended by the OS.
Maybe you can give an example of how to call testAsync so that thread 2000 returns to the thread pool?
Assuming thread 2k is the main thread, it cannot return to the thread pool. But you can use Task.Run(()=> foo()) to run foo() on the thread pool, and since the calling thread is the main thread, another thread pool thread will pick up that Task. So the following code:
static void Main(string[] args)
{
Console.WriteLine("main started on thread {0}", Thread.CurrentThread.ManagedThreadId);
var testAsyncTask = Task.Run(() => testAsync());
testAsyncTask.Wait();
}
static async Task testAsync()
{
Console.WriteLine("testAsync started on thread {0}", Thread.CurrentThread.ManagedThreadId);
await Task.Delay(1000);
Console.WriteLine("testAsync continued on thread {0}", Thread.CurrentThread.ManagedThreadId);
}
Produced (on my PC) the following output:
main started on thread 1
testAsync started on thread 3
testAsync continued on thread 4
Press any key to continue . . .
Threads 3 and 4 came from and returned to the thread pool.

You can print out the Thread.CurrentThread.ManagedThreadId to the console. Note that the thread-pool is free to re-use that same thread to run continuations on it, so there's no guarantee that it'll be different:
void Main()
{
TestAsync().Wait();
}
public async Task FooAsync()
{
int y = 0;
await Task.Delay(5);
Console.WriteLine($"After awaiting in FooAsync:
{Thread.CurrentThread.ManagedThreadId }");
while (y < 5) y++;
}
public async Task TestAsync()
{
Console.WriteLine($"Before awaiting in TestAsync:
{Thread.CurrentThread.ManagedThreadId }");
Task task = foo();
int i = 0;
while (i < 100)
{
var x = i++;
}
await task;
Console.WriteLine($"After awaiting in TestAsync:
{Thread.CurrentThread.ManagedThreadId }");
}
Another thing you can check is ThreadPool.GetAvailableThreads to determine if another worker has been handed out for use:
async Task FooAsync()
{
int y = 0;
await Task.Delay(5);
Console.WriteLine("Thread-Pool threads after first await:");
int avaliableWorkers;
int avaliableIo;
ThreadPool.GetAvailableThreads(out avaliableWorkers, out avaliableIo);
Console.WriteLine($"Available Workers: { avaliableWorkers},
Available IO: { avaliableIo }");
while (y < 1000000000) y++;
}
async Task TestAsync()
{
int avaliableWorkers;
int avaliableIo;
ThreadPool.GetAvailableThreads(out avaliableWorkers, out avaliableIo);
Console.WriteLine("Thread-Pool threads before first await:");
Console.WriteLine($"Available Workers: { avaliableWorkers},
Available IO: { avaliableIo }");
Console.WriteLine("-------------------------------------------------------------");
Task task = FooAsync();
int i = 0;
while (i < 100)
{
var x = i++;
}
await task;
}
On my machine, this yields:
Thread-Pool threads before first await:
Available Workers: 1023, Available IO: 1000
----------------------------------------------
Thread-Pool threads after first await:
Available Workers: 1022, Available IO: 1000

I'd like to verify which one is really happening.
There is no way to "verify" that with debugger, because the debugger is made to simulate the logical (synchronous) flow - see Walkthrough: Using the Debugger with Async Methods.
In order to understand what is happening (FYI it's your case (2)), you need to learn how await works starting from Asynchronous Programming with Async and Await - What Happens in an Async Method section, Control Flow in Async Programs and many other sources.
Look at this snippet:
static void Main(string[] args)
{
Task.Run(() =>
{
// Initial thread pool thread
var t = testAsync();
t.Wait();
});
Console.ReadLine();
}
If we make the lambda to be async and use await t; instead of t.Wait();, this is the point where the initial thread will be returned to the thread pool. As I mentioned above, you cannot verify that with debugger. But look at the above code and think logically - we are blocking the initial thread, so if it' wasn't free, your testAsync and foo methods will not be able to resume. But they do, and this can easily be verified by putting breakpoint after await lines.