We Keep Coding

What thread runs a Task's continuation if you don't await the task?

I'm trying to wrap my head around control flow in C# when using async, Task, and await.
I understand how promises work, and that the returned Task<> from an async method will eventually contain the result of a computation/IO/whatever.
I think I understand that if you explicitly wait for that Task, then the current thread blocks until the Task is complete. I also think that means that the code in the async method that returns a Task will be running on a thread in a thread pool.
What I don't understand is what happens if I don't "await" the Task returned by an asynchronous method. It seems to me that the continuation is executed on the original thread that calls the async method, but I have no idea how control can return to that thread.
Here's an example. Here's I'm using UniTask which is basically Tasks for Unity:
public async UniTask ConnectAsync(Connection connection)
{
Debug.Log(Thread.CurrentThread.Name); -> this prints "Main Thread"
// Close Any Old Connections
await DisconnectAsync();
// Default Address
if (string.IsNullOrEmpty(connection.Address)) { connection.Address = "localhost:6379"; }
// Connect
ConfigurationOptions config = new()
{
EndPoints =
{
{ connection.Address, connection.Port },
},
User = connection.Username,
Password = connection.Password,
};
m_Connection = await ConnectionMultiplexer.ConnectAsync(config);
// Create Graph Client
m_Graph = new(m_Connection.GetDatabase());
// Notify
await Editor.Controller.OnConnect();
Debug.Log(Thread.CurrentThread.Name); -> this prints "Main Thread"
}
If I call this method, and then neglect to await the returned Task (UniTask), both Debug.Log() show that execution is happening on the "Main Thread" (i.e. the UI thread).
How is it that without awaiting this Task, the Main Thread is able to return to this continuation? Does C# wait until the thread is in the Suspended/WaitSleepJoin state? I'm not aware of any code putting the UI thread to sleep so I'm not sure about that. I'm certainly not putting the UI to sleep.
EDIT: I believe the chosen answer basically answered the question in the final sentence:
"Your code just needs to return to the main loop to allow the
continuation to run."
In other words, there's a loop somewhere deep in the bowels of (Unity in this case) and if the UI thread gets there, then it takes the opportunity to continue any pending tasks. (Please correct me in a comment if this is wrong and I'll update accordingly).
Incidentally, these links were very informative:
https://blog.stephencleary.com/2013/11/there-is-no-thread.html
https://www.ncameron.org/blog/async-io-fundamentals/
https://devblogs.microsoft.com/pfxteam/executioncontext-vs-synchronizationcontext/
https://blog.stephencleary.com/2012/02/async-and-await.html

What I don't understand is what happens if I don't "await" the Task returned by an asynchronous method. It seems to me that the continuation is executed on the original thread that calls the async method, but I have no idea how control can return to that thread.
As I describe on my blog, each await (by default) captures a "context", which is SynchronizationContext.Current or TaskScheduler.Current. In this particular case, the UI context is captured and used to resume the async method (i.e., execute the continuation).
How is it that without awaiting this Task, the Main Thread is able to return to this continuation? Does C# wait until the thread is in the Suspended/WaitSleepJoin state?
It has to do with contexts, not threads. The UI context schedules work by posting to the main UI message queue. So the continuation is run when the UI thread processes its message queue; it doesn't have anything to do with thread states.
I'm not aware of any code putting the UI thread to sleep so I'm not sure about that. I'm certainly not putting the UI to sleep.
Your code just needs to return to the main loop to allow the continuation to run.

I understand how promises work
Good, then we can stop right there. Tasks are nothing but compiler syntactic sugar over a promise. In fact, when JavaScript copied the await/async keywords from C#, they got implemented over the native Promise object.
Now, for the remainder of this I'm going to assume that you don't know how promises work. Think of it as getting called out on your promise bluff on your CV.
There's three parts to an async method:
The "synchronous" part. This is what will run when you simply call your async function, awaiting it or not, and is everything before the first await in your function. In your function this is the Debug.Log call and the synchronous part of DisconnectAsync.
The "asynchronous" part, the tail of your function. This gets stored as a lambda and it captures all necessary variables on creation. This gets called after #1 and when "done" it returns the Task object from your function. When the task is fully completed, the task is set as completed. Note that this can be recursive if you have multiple tails inside your tail.
All the magic of Task. For example, Task.WhenAll instantiates mutexes in your Task and then waits on them for completion. This makes Task technically disposable, and thus a memory and OS handle leak if you don't dispose every single task you create. await itself is handled through TaskCompletionSource, and you get just the task it manages. Things like that.
Note that nowhere in this did I mention threads. Tasks are to threads like what cats are to doctors. They both exist, some interact, but you have to be pretty insane to say cats are made to work only with doctors. Instead, tasks work on contexts. Thread pools are one default context. Another is single threaded contexts.
That's right, you can easily have async code run on a single thread, which is perfect for GUI in a single threaded render loop-driven game. You create a dialog, await its showing and get a result, all without any additional threads. You start an animation and await its completion, all without any threads.

Would it be async if I call a number of async methods without awaits but then use Task.WhenAll?

I'm having a piece of code that I'm not quite sure if this would run asynchronously. Below I've made up some sample scripts which truly reflects the situation. Please note that the GetAsync methods are proper asyn methods having async/await keywords and return type using the Task of the related object.
public async Task<SomeResults> MyMethod()
{
var customers = _customerApi.GetAllAsync("some_url");
var orders = _orderApi.GetAllAsync("some_url");
var products = _productApi.GetAllAsync("some_url");
await Task.WhenAll(customers, orders, products);
// some more processing and returning the results
}
Question 1: Would the three above API calls run asynchronously even though there's no await before them? But, we have the await before Task.WhenAll?
Question 2: Would the above code run asynchronously if the await keyword is removed from before the Task.WhenAll?
I've tried to Google it around but couldn't find the proper answer to this specific situation. I've started reading Parallel Programming in Microsoft .NET but yet have long way to finish it so I couldn't just wait it.

Question 1: Would the three above API calls run asynchronously even though there's no await before them? But, we have the await before Task.WhenAll?
If the methods are actually doing something asynchronously, then yes.
Question 2: Would the above code run asynchronously if the await keyword is removed from before the Task.WhenAll?
If the methods are actually doing something asynchronously, then yes. However, it would be pointless to use Task.WhenAll without await.
Why I say "if": The async keyword doesn't magically make a method asynchronous, neither does the await operator. The methods still have to actually do something asynchronously. They do that by returning an incomplete Task.
All async methods start out running synchronously, just like any other method. The magic happens at await. If await is given an incomplete Task, then the method returns its own incomplete Task, with the rest of the method signed up as a continuation of that Task. That happens all the way up the call stack as long as you're using await all the way up the call stack.
Once the Task completes, then the continuation runs (the rest of the methods after await).
But at the top of the call stack needs to be something that's actually asynchronous. If you have an async method that calls an async method that calls a synchronous method, then nothing will actually run asynchronously, even if you use await.
For example, this will run completely synchronously (i.e. the thread will block) because an incomplete Task is never returned anywhere:
async Task Method1() {
await Method2();
}
async Task Method2() {
await Method3();
}
Task Method3() {
Thread.Sleep(2000);
return Task.CompletedTask;
}
However, this will run asynchronously (i.e. during the delay, the thread is freed to do other work):
async Task Method1() {
await Method2();
}
async Task Method2() {
await Method3();
}
async Task Method3() {
await Task.Delay(2000);
}
The key is in what Task.Delay returns. If you look at that source code, you can see it returns a DelayPromise (which inherits from Task), immediately (before the time is up). Since it's awaited, that triggers Method3 to return an incomplete Task. Since Method2 awaits that, it returns an incomplete Task, etc. all the way up the call stack.

YES, to both questions with a lot of caveats.
await / async is just syntactical sugar that allows you to write async code in a synchronous way. It doesn't magically spin up threads to make things run in parallel. It just allows the currently executing thread to be freed up to do other chunks of work.
Think of the await keyword as a pair of scissors that snips the current chunk of work into two, meaning the current thread can go and do another chunk while waiting for the result.
In order to do these chunks of work, there needs to be some kind of TaskScheduler. WinForms and WPF both provide TaskSchedulers that allow a single thread to process chunks one by one, but you can also use the default scheduler (via Task.Run()) which will use the thread pool, meaning lots of threads will run lots of chunks at once.
Assuming you are using a single thread, you example code would run as follows:
_customerApi.GetAllAsync() would run until it either completes, or hits an await. At that point it would return to a Task to your calling function which gets stuffed into customers.
_orderApi.GetAllAsync() would then run in exactly the same way. A Task will be assigned to orders which may or may not be complete.
ditto _productApi.GetAllAsync()
then you thread hits await Task.WhenAll(customers, orders, products); this means it can go and do other things, so the TaskScheduler might give it some other chunks of work to do, such as continuing to do the next bit of _customerApi.GetAllAsync().
Eventually all the chunks of work will be done, and your three tasks inside customers, orders, and products will be complete. At this point the scheduler knows that it can run the bit after WhenAll()
So you can see that in this case a SINGLE thread has run all the code, but not necessarily synchronously.

Whether your code runs asynchronously depends on your definition of asynchronous. If you look really close what happens, there will only be one thread that will do the stuff. However this thread won't be waiting idly as long as it has something to do.
A thing that really helped me to understand async-await was the cook-making-breakfast analogy in this interview with Eric Lippert. Search somewhere in the middle for async await.
Suppose a cook has to make breakfast. He starts boiling water for the tea. Instead of waiting idly for the water to cook, he inserts bread in the toaster. Not waiting idly again he starts boiling water for the eggs. As soon as the tea water boils he makes the tea and waits for the toast or the eggs.
Async-await is similar. Whenever your thread would have to wait for another process to finish, like a file to be written, a database query to return data, internet data to load, the thread won't be waiting idly for the other process to finish, but it will go up the call stack to see if any of the callers isn't awaiting and starts executing statements until it sees an await. Go up the call stack again and execute until the await. etc.
Because GetAllAsync is declared async, you can be certain that there is an await in it. In fact, your compiler will warn you if you declare a function async without an await in it.
Your thread will go into _customerApi.GetAllAsync("some_url"); and executes statements until is sees an await. If the task that your thread is awaiting for isn't complete, the thread goes up the call stack (your procedure) and starts executing the next statement:_orderApi.GetAllAsync("some_url"). It executes statements until is sees an await. Your function gets control again and calls the next method.
This goes on until your procedure starts awaiting. In this case, the awaitable method Task.WhenAll (not to be confused with the non-awaitable Task.WaitAll).
Even now, the thread won't be waiting idly, it will go up the call stack and execute statements until is meets an await, goes up the call stack again, etc.
So note: no new threads won't be started. While your thread is busy executing statements of the first method call, no statements of the second call will be executed, and while statements of the second call are being executed, no statements of the first call will be executed, not even if the first await is ready.
This is similar to the one and only cook: while he is inserting bread in the toaster, he can't process the boiling water for the tea: only after the bread is inserted and he would start waiting idly for it to be toasted he can continue making tea.
The await Task.WhenAll is not different to other awaits, except that the task is completed when all tasks are completed. So as long as any of the tasks is not ready, your thread won't execute statements after the WhenAll. However: your thread won't be waiting idly, it will go up the call stack and start executing statements.
So although it seems that two pieces of code are executed at the same time, it is not. If you really want two pieces of code to run simultaneously you'll have to hire a new cook using `Task.Run( () => SliceTomatoes);
Hiring a new cook (starting a new thread) is only meaningful if the other task is not async and your thread has other meaningful things to do, like keeping the UI responsive. Normally your cook would Slice the Tomatoes himself. Let your caller decide whether he hires a new cook (=you) to make the breakfast and slice the tomatoes.
I oversimplified it a bit, by telling you that there is only one thread (cook) involved. In fact, it can be any thread that continues executing the statements after your await. You can see that in the debugger by examining the thread ID, quite often it will be a different thread that will continue. However this thread has the same context as your original thread, so for you it will be as if it is the same thread: no need for a mutex, no need for IsInvokeRequired for user interface threads. More information about this can be found in articles from Stephen Cleary

C# Asynchronous call "invoke and forget"

In c# when you invoke the await method inside an async method, the code will be executed from the calling context to that method. What if I want to make an asynchronous call which will just continue from the same line of code directly (even if the asynchronous action did not finish yet) instead of getting back to the calling context? How can I do that?

What if I want to make an asynchronous call which will just continue from the same line of code directly (even if the asynchronous action did not finish yet) instead of getting back to the calling context? How can I do that?
The easiest way, which works whether Something is synchronous or asynchronous:
var _ = Task.Run(() => Something());
But, as Eric Lippert stated:
it is a strange thing to get a task and then not care about what happens when it completes.
Or, as I like to put it: fire-and-forget is almost never what you really want. To "forget" means:
You don't care when it completes.
You don't care whether it completes. E.g., if your app exits (or app pool is recycled if on ASP.NET), then your background work is just lost.
You don't care whether it completes successfully. E.g., if your work throws an exception, you want to silently swallow that exception.
In real-world code, this scenario is extremely rare.

An await is, by definition, a point at which you must asynchronously wait for the task to finish before continuing the workflow. If you don't have to wait, then don't wait! There is no requirement that you await anything.
Your question is rather like asking "I have an IEnumerable<int> but I don't care what integers are in it; do I have to foreach over it?" No, you don't. If you don't care what the result is, you don't have to get a result from a sequence. If you don't care what the result of a task is, you don't have to await it.
But it is a strange thing to get a sequence and then not enumerate it, and it is a strange thing to get a task and then not care about what happens when it completes.

If you want to start the async method and continue inside the function you can do something like Task<string> getStringTask = asyncMethodThatReturnsString();. Then continue until you want to wait for the async task to finish at which point you would call string output = await getStringTask;.

Async function with two awaited async calls, where the 2nd needs the results from the first

So I have a function that looks like:
async Task DoSomething() {
var result = await GetDataForAsyncCall2();
if (result != null) {
await AsyncCall2(result.result1, result.result2);
}
}
My problem is that whenever I try to call it, it seems to return from the function after the GetDataForAsyncCall2() is called. But the actual result I want from the function is AsyncCall2.
Why would my function be returning after the first await, or how can I ensure the second await is ran before the function returns?

Why would my function be returning after the first await?
The fact that you are asking the question indicates that you must have some completely false beliefs about what await does. This is like asking why return returns. An await is a kind of return. (Just like yield return is a kind of return in an iterator block. yield return and await are at some level essentially the same thing; they are both points at which a workflow returns to its caller and signs up the remainder of the method to run in the future.)
You probably need to do some research as to what await actually means in C#. Briefly, await means "if the results of this task are not available then return to my caller so that it can keep working. Pick up at this point some time in the future after the task I'm waiting for is complete." That is, asynchronously wait for the task to complete.
If this is the first await you hit in the method then the thing returned to the caller will be a task representing the method itself, because now it is not complete either, and the caller probably wants to await it.
But that brief explanation is likely not enough. You should read some articles or tutorials on how this works so that you can use it more effectively.
how can I ensure the second await is ran before the function returns?
You don't. That's by design. Awaits allow you to identify the points in an asynchronous workflow that are (1) high latency operations, and (2) where the operation must be completed before the remainder of the workflow executes. The sequence of awaits you have correctly represents the data dependency of the second high-latency task on the first. The system is working as designed.
But the actual result I want from the function is AsyncCall2.
I assume that the "result" you are after is a side effect since no value is extracted from AsyncCall2, and you have a valueless task returned from the method.
That's precisely what you get. Your method returns a task and that task will be marked as completed at some point in the future after the task returned by AsyncCall2 is completed. If your caller wants to asynchronously wait for that time in the future then it should await the returned task.
Again: an await is a point in an asynchronous workflow where we know that before it, the task has likely not completed, and after it, it definitely has completed.
A question that you did not ask:
Should I solve my problem by synchronously waiting for the high latency operation using .Result or similar?
No. That not only defeats the entire purpose of using await to manage latency. It also can cause you to wait forever. See http://blog.stephencleary.com/2012/07/dont-block-on-async-code.html for some examples.
These examples are highly educational; I would read them very carefully until you thoroughly understand how await works.

How does C# 5.0's async-await feature differ from the TPL?

I don't see the different between C#'s (and VB's) new async features, and .NET 4.0's Task Parallel Library. Take, for example, Eric Lippert's code from here:
async void ArchiveDocuments(List<Url> urls) {
Task archive = null;
for(int i = 0; i < urls.Count; ++i) {
var document = await FetchAsync(urls[i]);
if (archive != null)
await archive;
archive = ArchiveAsync(document);
}
}
It seems that the await keyword is serving two different purposes. The first occurrence (FetchAsync) seems to mean, "If this value is used later in the method and its task isn't finished, wait until it completes before continuing." The second instance (archive) seems to mean, "If this task is not yet finished, wait right now until it completes." If I'm wrong, please correct me.
Couldn't it just as easily be written like this?
void ArchiveDocuments(List<Url> urls) {
for(int i = 0; i < urls.Count; ++i) {
var document = FetchAsync(urls[i]); // removed await
if (archive != null)
archive.Wait(); // changed to .Wait()
archive = ArchiveAsync(document.Result); // added .Result
}
}
I've replaced the first await with a Task.Result where the value is actually needed, and the second await with Task.Wait(), where the wait is actually occurring. The functionality is (1) already implemented, and (2) much closer semantically to what is actually happening in the code.
I do realize that an async method is rewritten as a state machine, similar to iterators, but I also don't see what benefits that brings. Any code that requires another thread to operate (such as downloading) will still require another thread, and any code that doesn't (such as reading from a file) could still utilize the TPL to work with only a single thread.
I'm obviously missing something huge here; can anybody help me understand this a little better?

I think the misunderstanding arises here:
It seems that the await keyword is serving two different purposes. The first occurrence (FetchAsync) seems to mean, "If this value is used later in the method and its task isn't finished, wait until it completes before continuing." The second instance (archive) seems to mean, "If this task is not yet finished, wait right now until it completes." If I'm wrong, please correct me.
This is actually completely incorrect. Both of these have the same meaning.
In your first case:
var document = await FetchAsync(urls[i]);
What happens here, is that the runtime says "Start calling FetchAsync, then return the current execution point to the thread calling this method." There is no "waiting" here - instead, execution returns to the calling synchronization context, and things keep churning. At some point in the future, FetchAsync's Task will complete, and at that point, this code will resume on the calling thread's synchronization context, and the next statement (assigning the document variable) will occur.
Execution will then continue until the second await call - at which time, the same thing will happen - if the Task<T> (archive) isn't complete, execution will be released to the calling context - otherwise, the archive will be set.
In the second case, things are very different - here, you're explicitly blocking, which means that the calling synchronization context will never get a chance to execute any code until your entire method completes. Granted, there is still asynchrony, but the asynchrony is completely contained within this block of code - no code outside of this pasted code will happen on this thread until all of your code completes.

Anders boiled it down to a very succinct answer in the Channel 9 Live interview he did. I highly recommend it
The new Async and await keywords allow you to orchestrate concurrency in your applications. They don't actually introduce any concurrency in to your application.
TPL and more specifically Task is one way you can use to actually perform operations concurrently. The new async and await keyword allow you to compose these concurrent operations in a "synchronous" or "linear" fashion.
So you can still write a linear flow of control in your programs while the actual computing may or may not happen concurrently. When computation does happen concurrently, await and async allow you to compose these operations.

There is a huge difference:
Wait() blocks, await does not block. If you run the async version of ArchiveDocuments() on your GUI thread, the GUI will stay responsive while the fetching and archiving operations are running.
If you use the TPL version with Wait(), your GUI will be blocked.
Note that async manages to do this without introducing any threads - at the point of the await, control is simply returned to the message loop. Once the task being waited for has completed, the remainder of the method (continuation) is enqueued on the message loop and the GUI thread will continue running ArchiveDocuments where it left off.

The ability to turn the program flow of control into a state machine is what makes these new keywords intresting. Think of it as yielding control, rather than values.
Check out this Channel 9 video of Anders talking about the new feature.

The problem here is that the signature of ArchiveDocuments is misleading. It has an explicit return of void but really the return is Task. To me void implies synchronous as there is no way to "wait" for it to finish. Consider the alternate signature of the function.
async Task ArchiveDocuments(List<Url> urls) {
...
}
To me when it's written this way the difference is much more obvious. The ArchiveDocuments function is not one that completes synchronously but will finish later.

The await keyword does not introduce concurrency. It is like the yield keyword, it tells the compiler to restructure your code into lambda controlled by a state machine.
To see what await code would look like without 'await' see this excellent link: http://blogs.msdn.com/b/windowsappdev/archive/2012/04/24/diving-deep-with-winrt-and-await.aspx

The call to FetchAsync() will still block until it completes (unless a statement within calls await?) The key is that control is returned to the caller (because the ArchiveDocuments method itself is declared as async). So the caller can happily continue processing UI logic, respond to events, etc.
When FetchAsync() completes, it interrupts the caller to finish the loop. It hits ArchiveAsync() and blocks, but ArchiveAsync() probably just creates a new task, starts it, and returns the task. This allows the second loop to begin, while the task is processing.
The second loop hits FetchAsync() and blocks, returning control to the caller. When FetchAsync() completes, it again interrupts the caller to continue processing. It then hits await archive, which returns control to the caller until the Task created in loop 1 completes. Once that task is complete, the caller is again interrupted, and the second loop calls ArchiveAsync(), which gets a started task and begins loop 3, repeat ad nauseum.
The key is returning control to the caller while the heavy lifters are executing.