We Keep Coding

Why does an async Task.Run terminate a Thread and a Task Run does not?

I am starting a Thread where an await Task.Run can be invoked.
After starting a Thread with the ThreadStart.Start method, why does the await Task.Run terminate the Thread and Task.Run does not?
Here is some code as an example:
public async Task Task1()
{
if (awaitRunTask)
{
await Task.Run(async () =>
{
await Test();
}
);
}
else
{
Task.Run(async () =>
{
await Test();
}
);
}
}
In the above example, if a Thread invokes the Task1 method, and awaitRunTask = true, the Thread terminates. If awaitRunTask = false, the Thread does not terminate.
When I say terminate, the Thread does not complete correctly and the method where the ThreadStart.Start is invoked returns. This happens at the await Test() code.
Why is this and if I want to await a Task.Run on a Thread, is there a way to do this?
EDIT
Here is some code to show a more detailed example:
public class ThreadExample
{
public bool awaitRunTask;
public string value;
private async Task StartThreadAsync()
{
var method = this.GetType().GetMethod("RunTasksAsync");
ThreadStart threadStart;
threadStart = async () =>
{
await InvokeAsync(method, this, null);
};
var thread = new Thread(threadStart);
thread.Start();
thread.Join();
}
public async Task RunTasksAsync()
{
await Task1Async();
Task2();
}
private async Task Task1Async()
{
if (awaitRunTask)
{
await Task.Run(async () =>
{
await TestAsync();
}
);
}
else
{
Task.Run(async () =>
{
await TestAsync();
}
);
}
}
private void Task2()
{
value = "valid";
}
private async Task TestAsync()
{
await Task.Delay(1000);
}
private async Task InvokeAsync(MethodInfo method, object instance, object[] parameters)
{
dynamic awaitable = method.Invoke(instance, parameters);
await awaitable;
}
public async Task ValueIsCorrectAsync()
{
value = "not valid";
awaitRunTask = false;
await StartThreadAsync();
var isCorrect = (value == "valid");
}
public async Task ValueIsNotCorrectAsync()
{
value = "not valid";
awaitRunTask = true;
await StartThreadAsync();
var isCorrect = (value == "valid");
}
}
The ValueIsCorrectAsync method works correctly as the Task2 method sets the value field.
The ValueIsNotCorrectAsync method does not work correctly as the await Task.Run in the Task1Async method interferes with the Thread. The StartThreadAsync method returns before the Task2 method sets the value field.
The only difference between the two methods, is the value of awaitRunTask.
How should I change my code such that the value field is set correctly when awaitRunTask = true?
EDIT3
If the await Task.Delay(1000); is commented out in the TestAsync method, the code works for both awaitRunTask = true and awaitRunTask = false;
Can someone please explain to me why? I need to know why because the TestAsync method needs to be able to run asynchronous code.

Why is this?
As I explain on my blog, await here is actually returning to its caller. So the Task1 method returns an incomplete task to its caller, presumably the thread's main method, which presumably is async void. When the thread's main method returns (due to it's await), the thread exits.
The core of the problem is that the Thread type doesn't understand or work naturally with asynchronous code. Thread is really a very low-level building block at this point and is best avoided in modern code. There are very few scenarios where it can't be replaced with Task.Run.
if I want to await a Task.Run on a Thread, is there a way to do this?
The easiest solution is to get rid of the legacy thread completely; replace it with Task.Run.
Otherwise, you need the thread to block. If the continuations can run on thread pool threads, then you can just block directly (e.g., GetAwaiter().GetResult()). If the continuations need to run on that thread, then use AsyncContext from my AsyncEx library.

Here's a minimal version of your sample code:
async Task Main()
{
var te = new ThreadExample();
await te.StartThreadAsync(false);
await te.StartThreadAsync(true);
}
public class ThreadExample
{
public string value;
public async Task StartThreadAsync(bool awaitRunTask)
{
value = "not valid";
var thread = new Thread(() => Task1Async(awaitRunTask));
thread.Start();
thread.Join();
var isCorrect = (value == "valid");
Console.WriteLine(isCorrect);
}
private async Task Task1Async(bool awaitRunTask)
{
if (awaitRunTask)
{
await Task.Run(async () => await Task.Delay(1000));
}
value = "valid";
}
}
This outputs:
True
False
The thread that enters Task1Async executes the line value = "valid" when awaitRunTask == false, but when it's true it hits the await Task.Run and, because of the async state machine, the thread returns to the caller at this point and executes the thread.Join().
Effectively you've created an extreme race condition.

How to return AggregateException from async method

I got an async method working like an enhanced Task.WhenAll. It takes a bunch of tasks and returns when all are completed.
public async Task MyWhenAll(Task[] tasks) {
...
await Something();
...
// all tasks are completed
if (someTasksFailed)
throw ??
}
My question is how do I get the method to return a Task looking like the one returned from Task.WhenAll when one or more tasks has failed?
If I collect the exceptions and throw an AggregateException it will be wrapped in another AggregateException.
Edit: Full Example
async Task Main() {
try {
Task.WhenAll(Throw(1), Throw(2)).Wait();
}
catch (Exception ex) {
ex.Dump();
}
try {
MyWhenAll(Throw(1), Throw(2)).Wait();
}
catch (Exception ex) {
ex.Dump();
}
}
public async Task MyWhenAll(Task t1, Task t2) {
await Task.Delay(TimeSpan.FromMilliseconds(100));
try {
await Task.WhenAll(t1, t2);
}
catch {
throw new AggregateException(new[] { t1.Exception, t2.Exception });
}
}
public async Task Throw(int id) {
await Task.Delay(TimeSpan.FromMilliseconds(100));
throw new InvalidOperationException("Inner" + id);
}
For Task.WhenAll the exception is AggregateException with 2 inner exceptions.
For MyWhenAll the exception is AggregateException with one inner AggregateException with 2 inner exceptions.
Edit: Why I am doing this
I often need to call paging API:s and want to limit number of simultaneous connections.
The actual method signatures are
public static async Task<TResult[]> AsParallelAsync<TResult>(this IEnumerable<Task<TResult>> source, int maxParallel)
public static async Task<TResult[]> AsParallelUntilAsync<TResult>(this IEnumerable<Task<TResult>> source, int maxParallel, Func<Task<TResult>, bool> predicate)
It means I can do paging like this
var pagedRecords = await Enumerable.Range(1, int.MaxValue)
.Select(x => GetRecordsAsync(pageSize: 1000, pageNumber: x)
.AsParallelUntilAsync(maxParallel: 5, x => x.Result.Count < 1000);
var records = pagedRecords.SelectMany(x => x).ToList();
It all works fine, the aggregate within aggregate is just a minor inconvenience.

async methods are designed to only every set at most a single exception on the returned task, not multiple.
This leaves you with two options, you can either not use an async method to start with, instead relying on other means of performing your method:
public Task MyWhenAll(Task t1, Task t2)
{
return Task.Delay(TimeSpan.FromMilliseconds(100))
.ContinueWith(_ => Task.WhenAll(t1, t2))
.Unwrap();
}
If you have a more complex method that would be harder to write without using await, then you'll need to unwrap the nested aggregate exceptions, which is tedious, although not overly complex, to do:
public static Task UnwrapAggregateException(this Task taskToUnwrap)
{
var tcs = new TaskCompletionSource<bool>();
taskToUnwrap.ContinueWith(task =>
{
if (task.IsCanceled)
tcs.SetCanceled();
else if (task.IsFaulted)
{
if (task.Exception is AggregateException aggregateException)
tcs.SetException(Flatten(aggregateException));
else
tcs.SetException(task.Exception);
}
else //successful
tcs.SetResult(true);
});
IEnumerable<Exception> Flatten(AggregateException exception)
{
var stack = new Stack<AggregateException>();
stack.Push(exception);
while (stack.Any())
{
var next = stack.Pop();
foreach (Exception inner in next.InnerExceptions)
{
if (inner is AggregateException innerAggregate)
stack.Push(innerAggregate);
else
yield return inner;
}
}
}
return tcs.Task;
}

Use a TaskCompletionSource.
The outermost exception is created by .Wait() or .Result - this is documented as wrapping the exception stored inside the Task inside an AggregateException (to preserve its stack trace - this was introduced before ExceptionDispatchInfo was created).
However, Task can actually contain many exceptions. When this is the case, .Wait() and .Result will throw an AggregateException which contains multiple InnerExceptions. You can access this functionality through TaskCompletionSource.SetException(IEnumerable<Exception> exceptions).
So you do not want to create your own AggregateException. Set multiple exceptions on the Task, and let .Wait() and .Result create that AggregateException for you.
So:
var tcs = new TaskCompletionSource<object>();
tcs.SetException(new[] { t1.Exception, t2.Exception });
return tcs.Task;
Of course, if you then call await MyWhenAll(..) or MyWhenAll(..).GetAwaiter().GetResult(), then it will only throw the first exception. This matches the behaviour of Task.WhenAll.
This means you need to pass tcs.Task up as your method's return value, which means your method can't be async. You end up doing ugly things like this (adjusting the sample code from your question):
public static Task MyWhenAll(Task t1, Task t2)
{
var tcs = new TaskCompletionSource<object>();
var _ = Impl();
return tcs.Task;
async Task Impl()
{
await Task.Delay(10);
try
{
await Task.WhenAll(t1, t2);
tcs.SetResult(null);
}
catch
{
tcs.SetException(new[] { t1.Exception, t2.Exception });
}
}
}
At this point, though, I'd start to query why you're trying to do this, and why you can't use the Task returned from Task.WhenAll directly.

I deleted my previous answer, because I found a simpler solution. This solution does not involve the pesky ContinueWith method or the TaskCompletionSource type. The idea is to return a nested Task<Task> from a local function, and Unwrap() it from the outer container function. Here is a basic outline of this idea:
public Task<T[]> GetAllAsync<T>()
{
return LocalAsyncFunction().Unwrap();
async Task<Task<T[]>> LocalAsyncFunction()
{
var tasks = new List<Task<T>>();
// ...
await SomethingAsync();
// ...
Task<T[]> whenAll = Task.WhenAll(tasks);
return whenAll;
}
}
The GetAllAsync method is not async. It delegates all the work to the LocalAsyncFunction, which is async, and then Unwraps the resulting nested task and returns it. The unwrapped task contains in its .Exception.InnerExceptions property all the exceptions of the tasks, because it is just a facade of the internal Task.WhenAll task.
Let's demonstrate a more practical realization of this idea. The AsParallelUntilAsync method below enumerates lazily the source sequence and projects the items it contains to Task<TResult>s, until an item satisfies the predicate. It also limits the concurrency of the asynchronous operations. The difficulty is that enumerating the IEnumerable<TSource> could throw an exception too. The correct behavior in this case is to await all the running tasks before propagating the enumeration error, and return an AggregateException that contains both the enumeration error, and all the task errors that may have occurred in the meantime. Here is how it can be done:
public static Task<TResult[]> AsParallelUntilAsync<TSource, TResult>(
this IEnumerable<TSource> source, Func<TSource, Task<TResult>> action,
Func<TSource, bool> predicate, int maxConcurrency)
{
return Implementation().Unwrap();
async Task<Task<TResult[]>> Implementation()
{
var tasks = new List<Task<TResult>>();
async Task<TResult> EnumerateAsync()
{
var semaphore = new SemaphoreSlim(maxConcurrency, maxConcurrency);
using var enumerator = source.GetEnumerator();
while (true)
{
await semaphore.WaitAsync();
if (!enumerator.MoveNext()) break;
var item = enumerator.Current;
if (predicate(item)) break;
async Task<TResult> RunAndRelease(TSource item)
{
try { return await action(item); }
finally { semaphore.Release(); }
}
tasks.Add(RunAndRelease(item));
}
return default; // A dummy value that will never be returned
}
Task<TResult> enumerateTask = EnumerateAsync();
try
{
await enumerateTask; // Make sure that the enumeration succeeded
Task<TResult[]> whenAll = Task.WhenAll(tasks);
await whenAll; // Make sure that all the tasks succeeded
return whenAll;
}
catch
{
// Return a faulted task that contains ALL the errors!
return Task.WhenAll(tasks.Prepend(enumerateTask));
}
}
}

ContinueWith and TaskCancellation - How to return default-values if task fails?

I read a few threads about TaskCancellations.. However, I cannot find a solution for a simple question: How do I get a default value when my task fails?
I cannot (!) modify the task itself and put a try catch wrapper around it. I could of course put a try-catch around await, but I would like to handle this with ContinueWith - if possible.
public Task<List<string>> LoadExample()
{
Task<List<string>> task = LoadMyExampleTask();
task.ContinueWith(t => default(List<string>), TaskContinuationOptions.OnlyOnFaulted);
return task;
}
I thought this would be the correct way to deal with the problem. However, my application throws a JsonParseException (which is called in LoadMyExampleTask). I would expect to get null or (even better) an empty list.
In fact, all I want is:
var emptyOrFilledList = await LoadExample(); // guaranteed no exception thrown
Based on Luaan's great answer I wrote an extension method with a defaultValue-option:
public static Task<T> DefaultIfFaulted<T>(this Task<T> #this, T defaultValue = default(T))
{
return #this.ContinueWith(t => t.IsCompleted ? t.Result : defaultValue);
}
Edit: await myTask.DefaultifFaulted() just throwed a
[ERROR] FATAL UNHANDLED EXCEPTION: System.AggregateException
Are you sure that every exception is caught?

If you want that, you must not return the original task - you need to return the continuation.
public Task<List<string>> LoadExample()
{
Task<List<string>> task = LoadMyExampleTask();
return task.ContinueWith(t =>
t.IsFaulted || t.IsCanceled ? default(List<string>) : t.Result);
}
Your original code did allow the continuation to run when the original task faulted, but you didn't read the status of that task - the fact that a task has a continuation which handles errors is entirely irrelevant to what an await on the original task will do.
Of course, it's rather easy to make this into a generic helper method:
public static Task<T> DefaultIfFaulted<T>(this Task<T> #this)
{
return #this.ContinueWith (t => t.IsCanceled || t.IsFaulted ? default(T) : t.Result);
}

As promised, here are the DefaultIfFaulted<T> variants which are true to their name (and the title of this question). They preserve the antecedent task's behavior unless it's faulted (specifically, cancellation is propagated rather than ignored or masked by an AggregateException):
Old-school (.NET 4.0) way:
public static Task<T> DefaultIfFaulted<T>(this Task<T> task)
{
// The continuation simply returns the antecedent task unless it's faulted.
Task<Task<T>> continuation = task.ContinueWith(
t => (t.Status == TaskStatus.Faulted) ? Task.FromResult(default(T)) : t,
TaskContinuationOptions.ExecuteSynchronously
);
return continuation.Unwrap();
}
Async/await way (simple but slower):
public static async Task<T> DefaultIfFaulted<T>(this Task<T> task)
{
try
{
return await task.ConfigureAwait(false);
}
catch (Exception ex) when (!(ex is OperationCanceledException))
{
return default(T);
}
}
Async/await way (perf almost identical to Unwrap):
public static async Task<T> DefaultIfFaulted<T>(this Task<T> task)
{
// Await completion regardless of resulting Status (alternatively you can use try/catch).
await task
.ContinueWith(_ => { }, TaskContinuationOptions.ExecuteSynchronously)
.ConfigureAwait(false);
return task.Status != TaskStatus.Faulted
// This await preserves the task's behaviour
// in all cases other than faulted.
? await task.ConfigureAwait(continueOnCapturedContext: false)
: default(T);
}
Tests (passed by all of the above):
using Xunit;
[Fact]
public async Task DefaultIfFaultedTest()
{
var success = Task.Run(() => 42);
var faulted = Task.Run(new Func<int>(() => { throw new InvalidOperationException(); }));
Assert.Equal(42, await success.DefaultIfFaulted());
Assert.Equal(0, await faulted.DefaultIfFaulted());
await Assert.ThrowsAsync<TaskCanceledException>(() =>
{
var tcs = new TaskCompletionSource<int>();
tcs.SetCanceled();
return tcs.Task.DefaultIfFaulted();
});
}

Async/Await behaviour through the stack

I'm curious about how the flow of async works across the stack. When reading about async in C#, you will continually read some version of the following:
If the task we are awaiting has not yet completed then sign up the
rest of this method as the continuation of that task, and then return
to your caller immediately; the task will invoke the continuation when
it completes.
It's the return to your caller immediately part that confuses me. In the below example, assuming something calls MethodOneAsync(), execution hits the await in MethodOneAsync. Does it immediately return to the method that called this? If so, how does MethodTwoAsync ever get executed? Or does it continue down the stack until it detects that it's actually blocked (ie. DoDBWorkAsync()) and then yield the thread?
public static async Task MethodOneAsync()
{
DoSomeWork();
await MethodTwoAsync();
}
public static async Task MethodTwoAsync()
{
DoSomeWork();
await MethodThreeAsync();
}
public static async Task MethodThreeAsync()
{
DoSomeWork();
await DoDBWorkAsync();
}

The part before an await in an async method is executed synchronously. That's the case for all async methods.
Let's assume that instead of await DoDBWorkAsync() we have await Task.Delay(1000).
That means MethodOneAsync starts running, executes DoSomeWork and calls MethodTwoAsync which in turn executes DoSomeWork which calls MethodThreeAsync which again executes DoSomeWork.
Then it calls Task.Delay(1000), gets back an uncompleted Task and awaits it.
That await is logically equivalent to adding a continuation and returning the task back to the caller, which is MethodTwoAsync which does the same and return a Task to the caller and so forth and so forth.
That way when the root delay Task completes all the continuations can run one after the other.
If we make your example a bit more complicated:
public static async Task MethodOneAsync()
{
DoSomeWorkOne();
await MethodTwoAsync();
DoMoreWorkOne();
}
public static async Task MethodTwoAsync()
{
DoSomeWorkTwo();
await MethodThreeAsync();
DoMoreWorkTwo();
}
public static async Task MethodThreeAsync()
{
DoSomeWorkThree();
await Task.Delay(1000);
DoMoreWorkThree();
}
It would be logically similar to doing this with continuations:
public static Task MethodOneAsync()
{
DoSomeWorkOne();
DoSomeWorkTwo();
DoSomeWorkThree();
return Task.Delay(1000).
ContinueWith(_ => DoMoreWorkThree()).
ContinueWith(_ => DoMoreWorkTwo()).
ContinueWith(_ => DoMoreWorkOne());
}

First, let me do a different example so my code further down will make sense
public static async Task MethodOneAsync()
{
DoSomeWork1();
await MethodTwoAsync();
DoOtherWork1();
}
public static async Task MethodTwoAsync()
{
DoSomeWork2();
await MethodThreeAsync();
DoOtherWork2();
}
public static async Task MethodThreeAsync()
{
DoSomeWork3();
await DoDBWorkAsync();
DoOtheWork3();
}
All async await does is turn the above code in to something similar to (but even this is a HUGE simplification) this
public static Task MethodOneAsync()
{
DoSomeWork1();
var syncContext = SynchronizationContext.Current ?? new SynchronizationContext();
var resultTask = MethodTwoAsync();
return resultTask.ContinueWith((task) =>
{
syncContext.Post((state) =>
{
SynchronizationContext.SetSynchronizationContext(syncContext);
DoOtherWork1();
}, null);
});
}
public static Task MethodTwoAsync()
{
DoSomeWork2();
var syncContext = SynchronizationContext.Current ?? new SynchronizationContext();
var resultTask = MethodThreeAsync();
return resultTask.ContinueWith((task) =>
{
syncContext.Post((state) =>
{
SynchronizationContext.SetSynchronizationContext(syncContext);
DoOtherWork2();
}, null);
});
}
public static Task MethodThreeAsync()
{
DoSomeWork3();
var syncContext = SynchronizationContext.Current ?? new SynchronizationContext();
var resultTask = DoDbWorkAsync();
return resultTask.ContinueWith((task) =>
{
syncContext.Post((state) =>
{
SynchronizationContext.SetSynchronizationContext(syncContext);
DoOtherWork3();
}, null);
});
}.
Each await just executes the next layer deeper till it is forced to return a Task, once that happens it starts doing continuations on the tasks when they complete and in that continuation it passes in a delegate representing "the rest of the function" to SynchronizationContext.Post( to get the code scheduled to be executed.
How it is scheduled depends on which SynchronizationContext you are in. In WPF and Winforms it queues it to the message pump, for the default and ASP.NET it queues it on a thread pool worker.

It does return after firing the Task for MethodTwoAsync method (or continues execution, if this Task is immediately done), so the inner Task is executing into SynchronizationContext.Current environment.
And after this Task is done, the .NET state machine return the execution to the point right after the MethodTwoAsync firing, and process the rest of the code.
See more information at MSDN article:

Why does this code ignore the await and proceed anyway?

I have the following code:
public Index () {
InitializeIndexAsync();
}
async Task InitializeIndexAsync () {
State = IndexState.Initializing;
await Task.Factory.StartNew(async () => {
// Initialize other things.
await IndexAsync();
});
State = IndexState.Ready;
}
I would expect that "State = IndexState.Ready" would not be hit until the asynchronous lambda completes, but debugging shows that line is hit long before the thread started above it completes. Why is this?

StartNew does not understand async lambdas, so when you pass it an async lambda, it will return a Task<Task>. Conceptually, the "outer" task only represents the start of the async lambda; the "inner" task represents the completion of the async lambda.
This is one of the reasons that StartNew is the wrong choice for async code, as I explain on my blog. A better solution is to use Task.Run, which was designed with async in mind:
async Task InitializeIndexAsync () {
State = IndexState.Initializing;
await Task.Run(async () => {
// Initialize other things.
await IndexAsync();
});
State = IndexState.Ready;
}

Not sure what you are trying to achieve by all these awaits...
I would try to keep it simple by having a synchronously method which initializes things, and then another MethodAsync which returns a Task and I can await on that task:
public async void Index()
{
await InitializeIndexAsync();
}
private Task InitializeIndexAsync()
{
return Task.Factory.StartNew(() => InitializeIndex());
}
private void InitializeIndex()
{
State = IndexState.Initializing;
// Initialize other things synchronously.
IndexAsync().Wait();
State = IndexState.Ready;
}
I hope this is what you meant.