We Keep Coding

Async/await: who is waiting/blocking?

I'm trying to understand async/await and read the source code of AsyncMethodBuilder. I thought there must be some code like xxx.Wait() or xxx.WaitOnce() waiting for a task to be completed.
However, I didn't find such code in class AsyncMethodBuilder.
system\runtime\compilerservices\AsyncMethodBuilder.cs
https://referencesource.microsoft.com/#mscorlib/system/runtime/compilerservices/AsyncMethodBuilder.cs,96
So I keep digging, tried to read the source code of Task, TaskScheduler, ThreadPoolTaskScheduler, ThreadPool.
Finally I got class _ThreadPoolWaitCallback, but didn't find any caller.
https://referencesource.microsoft.com/#mscorlib/system/threading/threadpool.cs,d7b8a78b4dd14fd0
internal static class _ThreadPoolWaitCallback
{
[System.Security.SecurityCritical]
static internal bool PerformWaitCallback()
{
return ThreadPoolWorkQueue.Dispatch();
}
}
Another possible code is in class SynchronizationContext, method SetWaitNotificationRequired()
https://referencesource.microsoft.com/#mscorlib/system/threading/synchronizationcontext.cs,8b34a86241c7b423
protected void SetWaitNotificationRequired()
{
...
RuntimeHelpers.PrepareDelegate(new WaitDelegate(this.Wait));
...
But I didn't know what RuntimeHelpers.PrepareDelegate is doing, which is a native method.
Please give some advice. Is there a Wait? And if it is, where it is?

In a correctly implemented async implementation: there is no wait. Instead, at the bottom of the chain, some code exists that will create some async source, which could be a TaskCompletionSource<T>, an IValueTaskSource[<T>], or something similar - which allows that code to store that token somewhere (for example, in a queue, a correlation dictionary, or an async state object for IOCP), and return the incomplete task to the caller. The caller then discovers that it is incomplete, and registers a "when you have the answer, do this to reactivate me" callback. That calling code now unwinds completely, with every step saying "when you're done, push here", and the thread goes on to do other things, such as service a different request.
At some point in the future (hopefully), the result will come back - again, via IOCP, or via a separate IO reader pulling a response from somewhere and taking the appropriate item out of the queue/correlation-dictionary, and says "the outcome was {...}" (TrySetResult, TrySetException, etc).
For all of that time no threads were blocked. That is, ultimately, the entire point of async/await: to free up threads, to increase scalability.
In incorrectly implemented async systems: anything and everything is possible, including async-over-sync, sync-over-async, and everything else.

Task.WhenAny does not exist when task is finished

We have a logic for a background job to keep running, either every 20 minuts, or when a task if finished.
A simplified version of what I want to do is as followed:
Task to control if we need to exit:
private static TaskCompletionSource<bool> forceSyncTask = new TaskCompletionSource<bool>();
Background job:
Task.Factory.StartNew(
async () =>
{
do
{
await Dosomething();
await Task.WhenAny(Task.Delay(TimeSpan.FromMinutes(20)), forceSyncTask.Task);
// Always reset the force sync property
forceSyncTask = new TaskCompletionSource<bool>();
}
while (true);
});
Then everytime there is a notification comes, I run the following to force to exit the Task.WhenAny
if (!forceSyncTask.Task.IsCompleted)
{
forceSyncTask.TrySetResult(true);
}
I tested it in dev box and it works. However after I deployed it to our webervice in prod environment,
even if I successfully SetResult (I have logging to know if TrySetResult returns true or not), the Task.WhenAny does not exit as expected.
Anyone has any idea why?

First, I recommend you use an established solution for pausing asynchronous methods, such as Stephen Toub's PauseToken or the PauseToken from my AsyncEx library. There's some red flags in the code as it currently stands: StartNew with an async delegate and without a TaskScheduler, and TaskCompletionSource<T> being used without the RunContinuationsAsynchronously option. It's better to stick to higher-level constructs (Task.Run and PauseToken, respectively) because there are lots of sharp corners on the low-level constructs.
As far as what exactly the problem is, that's difficult to tell, especially since you (and we) cannot reproduce it locally. Here's my top guesses:
You're running into a problem caused by the fact that continuations run synchronously if possible - i.e., TrySetResult ends up directly invoking some code within Task.WhenAny.
You're experiencing thread exhaustion on your production server.

Manually capturing and applying SynchronizationContext when completing a Task

I was having a problem with a hanging await (described here). During research I found out that calling SetResult on my TaskCompletionSource actually invokes awaiting continuation in the context of the thread that called SetResult (this is also spelled out in this answer to a somewhat related question). In my case this is a different thread (a thread-pool worker thread) from the one that started the await (an ASP.NET request thread).
While I'm still not sure why this would cause a hang, I decided to try forcing the SetResult into the original context. I stored the value of SynchronizationContext.Current before entering await on the request thread and manually applied it in the worker thread via SynchronizationContext.SetSynchronizationContext just before calling SetResult. This solved the hang and I can now await all my async methods without having to specify ConfigureAwait(false).
My question is: is this a reasonable and correct approach to manually capturing and applying the SynchronizationContext? FWIW, I tried doing a simple Post() with the SetResult delegate first, but that still caused a hang. I'm obviously a bit out of my comfort zone here... Please help me understand what's going on!

SetResult is not guaranteed to call anything. Therefore, this is not reliable.
You need to switch the sync context at the point where it is captured. A common pain point here is WebClient which captures the context when starting a web request. So your code would look like this:
SetContext(newContext);
new WebClient().DownloadAsync(...);
SetContext(oldContext);
Restore the old context to not disturb anything.
In other words the problem is in the continuation code, not in the code calling SetResult.

To my embarrassment, I had completely overlooked that my HTTP handler was derived from a small base class, which implemented IAsyncHttpHandler in a very questionable way in order to add support for async handlers:
public IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData)
{
...
var task = HandleRequestAsync(...);
Task.Run(async () => { await task; }).GetAwaiter().GetResult();
...
}
I can't even remember why I did this in the first place (it was over a year ago), but it definitely was THE stupid part I was looking for for the last couple of days!
Changing the handler base class to .NET 4.6's HttpTaskAsyncHandler got rid of the hangs. Sorry for wasting everyone's time! :(

Synchronous I/O within an async/await-based Windows Service

Let's say I have a Windows Service which is doing some bit of work, then sleeping for a short amount of time, over and over forever (until the service is shut down). So in the service's OnStart, I could start up a thread whose entry point is something like:
private void WorkerThreadFunc()
{
while (!shuttingDown)
{
DoSomething();
Thread.Sleep(10);
}
}
And in the service's OnStop, I somehow set that shuttingDown flag and then join the thread. Actually there might be several such threads, and other threads too, all started in OnStart and shut down/joined in OnStop.
If I want to instead do this sort of thing in an async/await based Windows Service, it seems like I could have OnStart create cancelable tasks but not await (or wait) on them, and have OnStop cancel those tasks and then Task.WhenAll().Wait() on them. If I understand correctly, the equivalent of the "WorkerThreadFunc" shown above might be something like:
private async Task WorkAsync(CancellationToken cancel)
{
while (true)
{
cancel.ThrowIfCancellationRequested();
DoSomething();
await Task.Delay(10, cancel).ConfigureAwait(false);
}
}
Question #1: Uh... right? I am new to async/await and still trying to get my head around it.
Assuming that's right, now let's say that DoSomething() call is (or includes) a synchronous write I/O to some piece of hardware. If I'm understanding correctly:
Question #2: That is bad? I shouldn't be doing synchronous I/O within a Task in an async/await-based program? Because it ties up a thread from the thread pool while the I/O is happening, and threads from the thread pool are a highly limited resource? Please note that I might have dozens of such Workers going simultaneously to different pieces of hardware.
I am not sure I'm understanding that correctly - I am getting the idea that it's bad from articles like Stephen Cleary's "Task.Run Etiquette Examples: Don't Use Task.Run for the Wrong Thing", but that's specifically about it being bad to do blocking work within Task.Run. I'm not sure if it's also bad if I'm just doing it directly, as in the "private async Task Work()" example above?
Assuming that's bad too, then if I understand correctly I should instead utilize the nonblocking version of DoSomething (creating a nonblocking version of it if it doesn't already exist), and then:
private async Task WorkAsync(CancellationToken cancel)
{
while (true)
{
cancel.ThrowIfCancellationRequested();
await DoSomethingAsync(cancel).ConfigureAwait(false);
await Task.Delay(10, cancel).ConfigureAwait(false);
}
}
Question #3: But... what if DoSomething is from a third party library, which I must use and cannot alter, and that library doesn't expose a nonblocking version of DoSomething? It's just a black box set in stone that at some point does a blocking write to a piece of hardware.
Maybe I wrap it and use TaskCompletionSource? Something like:
private async Task WorkAsync(CancellationToken cancel)
{
while (true)
{
cancel.ThrowIfCancellationRequested();
await WrappedDoSomething().ConfigureAwait(false);
await Task.Delay(10, cancel).ConfigureAwait(false);
}
}
private Task WrappedDoSomething()
{
var tcs = new TaskCompletionSource<object>();
DoSomething();
tcs.SetResult(null);
return tcs.Task;
}
But that seems like it's just pushing the issue down a bit further rather than resolving it. WorkAsync() will still block when it calls WrappedDoSomething(), and only get to the "await" for that after WrappedDoSomething() has already completed the blocking work. Right?
Given that (if I understand correctly) in the general case async/await should be allowed to "spread" all the way up and down in a program, would this mean that if I need to use such a library, I essentially should not make the program async/await-based? I should go back to the Thread/WorkerThreadFunc/Thread.Sleep world?
What if an async/await-based program already exists, doing other things, but now additional functionality that uses such a library needs to be added to it? Does that mean that the async/await-based program should be rewritten as a Thread/etc.-based program?

Actually there might be several such threads, and other threads too, all started in OnStart and shut down/joined in OnStop.
On a side note, it's usually simpler to have a single "master" thread that will start/join all the others. Then OnStart/OnStop just deals with the master thread.
If I want to instead do this sort of thing in an async/await based Windows Service, it seems like I could have OnStart create cancelable tasks but not await (or wait) on them, and have OnStop cancel those tasks and then Task.WhenAll().Wait() on them.
That's a perfectly acceptable approach.
If I understand correctly, the equivalent of the "WorkerThreadFunc" shown above might be something like:
Probably want to pass the CancellationToken down; cancellation can be used by synchronous code, too:
private async Task WorkAsync(CancellationToken cancel)
{
while (true)
{
DoSomething(cancel);
await Task.Delay(10, cancel).ConfigureAwait(false);
}
}
Question #1: Uh... right? I am new to async/await and still trying to get my head around it.
It's not wrong, but it only saves you one thread on a Win32 service, which doesn't do much for you.
Question #2: That is bad? I shouldn't be doing synchronous I/O within a Task in an async/await-based program? Because it ties up a thread from the thread pool while the I/O is happening, and threads from the thread pool are a highly limited resource? Please note that I might have dozens of such Workers going simultaneously to different pieces of hardware.
Dozens of threads are not a lot. Generally, asynchronous I/O is better because it doesn't use any threads at all, but in this case you're on the desktop, so threads are not a highly limited resource. async is most beneficial on UI apps (where the UI thread is special and needs to be freed), and ASP.NET apps that need to scale (where the thread pool limits scalability).
Bottom line: calling a blocking method from an asynchronous method is not bad but it's not the best, either. If there is an asynchronous method, call that instead. But if there isn't, then just keep the blocking call and document it in the XML comments for that method (because an asynchronous method blocking is rather surprising behavior).
I am getting the idea that it's bad from articles like Stephen Cleary's "Task.Run Etiquette Examples: Don't Use Task.Run for the Wrong Thing", but that's specifically about it being bad to do blocking work within Task.Run.
Yes, that is specifically about using Task.Run to wrap synchronous methods and pretend they're asynchronous. It's a common mistake; all it does is trade one thread pool thread for another.
Assuming that's bad too, then if I understand correctly I should instead utilize the nonblocking version of DoSomething (creating a nonblocking version of it if it doesn't already exist)
Asynchronous is better (in terms of resource utilization - that is, fewer threads used), so if you want/need to reduce the number of threads, you should use async.
Question #3: But... what if DoSomething is from a third party library, which I must use and cannot alter, and that library doesn't expose a nonblocking version of DoSomething? It's just a black box set in stone that at some point does a blocking write to a piece of hardware.
Then just call it directly.
Maybe I wrap it and use TaskCompletionSource?
No, that doesn't do anything useful. That just calls it synchronously and then returns an already-completed task.
But that seems like it's just pushing the issue down a bit further rather than resolving it. WorkAsync() will still block when it calls WrappedDoSomething(), and only get to the "await" for that after WrappedDoSomething() has already completed the blocking work. Right?
Yup.
Given that (if I understand correctly) in the general case async/await should be allowed to "spread" all the way up and down in a program, would this mean that if I need to use such a library, I essentially should not make the program async/await-based? I should go back to the Thread/WorkerThreadFunc/Thread.Sleep world?
Assuming you already have a blocking Win32 service, it's probably fine to just keep it as it is. If you are writing a new one, personally I would make it async to reduce threads and allow asynchronous APIs, but you don't have to do it either way. I prefer Tasks over Threads in general, since it's much easier to get results from Tasks (including exceptions).
The "async all the way" rule only goes one way. That is, once you call an async method, then its caller should be async, and its caller should be async, etc. It does not mean that every method called by an async method must be async.
So, one good reason to have an async Win32 service would be if there's an async-only API you need to consume. That would cause your DoSomething method to become async DoSomethingAsync.
What if an async/await-based program already exists, doing other things, but now additional functionality that uses such a library needs to be added to it? Does that mean that the async/await-based program should be rewritten as a Thread/etc.-based program?
No. You can always just block from an async method. With proper documentation so when you are reusing/maintaining this code a year from now, you don't swear at your past self. :)

If you still spawn your threads, well, yes, it's bad. Because it will not give you any benefit as the thread is still allocated and consuming resources for the specific purpose of running your worker function. Running a few threads to be able to do work in parallel within a service has a minimal impact on your application.
If DoSomething() is synchronous, you could switch to the Timer class instead. It allows multiple timers to use a smaller amount of threads.
If it's important that the jobs can complete, you can modify your worker classes like this:
SemaphoreSlim _shutdownEvent = new SemaphoreSlim(0,1);
public async Task Stop()
{
return await _shutdownEvent.WaitAsync();
}
private void WorkerThreadFunc()
{
while (!shuttingDown)
{
DoSomething();
Thread.Sleep(10);
}
_shutdownEvent.Release();
}
.. which means that during shutdown you can do this:
var tasks = myServices.Select(x=> x.Stop());
Task.WaitAll(tasks);

A thread can only do one thing at a time. While it is working on your DoSomething it can't do anything else.
In an interview Eric Lippert described async-await in a restaurant metaphor. He suggests to use async-await only for functionality where your thread can do other things instead of waiting for a process to complete, like respond to operator input.
Alas, your thread is not waiting, it is doing hard work in DoSomething. And as long as DoSomething is not awaiting, your thread will not return from DoSomething to do the next thing.
So if your thread has something meaningful to do while procedure DoSomething is executing, it's wise to let another thread do the DoSomething, while your original thread is doing the meaningful stuff. Task.Run( () => DoSomething()) could do this for you. As long as the thread that called Task.Run doesn't await for this task, it is free to do other things.
You also want to cancel your process. DoSomething can't be cancelled. So even if cancellation is requested you'll have to wait until DoSomething is completed.
Below is your DoSomething in a form with a Start button and a Cancel button. While your thread is DoingSomething, one of the meaningful things your GUI thread may want to do is respond to pressing the cancel button:
void CancellableDoSomething(CancellationToken token)
{
while (!token.IsCancellationRequested)
{
DoSomething()
}
}
async Task DoSomethingAsync(CancellationToken token)
{
var task = Task.Run(CancellableDoSomething(token), token);
// if you have something meaningful to do, do it now, otherwise:
return Task;
}
CancellationTokenSource cancellationTokenSource = null;
private async void OnButtonStartSomething_Clicked(object sender, ...)
{
if (cancellationTokenSource != null)
// already doing something
return
// else: not doing something: start doing something
cancellationTokenSource = new CancellationtokenSource()
var task = AwaitDoSomethingAsync(cancellationTokenSource.Token);
// if you have something meaningful to do, do it now, otherwise:
await task;
cancellationTokenSource.Dispose();
cancellationTokenSource = null;
}
private void OnButtonCancelSomething(object sender, ...)
{
if (cancellationTokenSource == null)
// not doing something, nothing to cancel
return;
// else: cancel doing something
cancellationTokenSource.Cancel();
}

Akavache's GetObject<T> hangs when awaited. Any idea what is wrong here?

I have a Xamarin.Forms application, with this code in my App class (yes, this is just a sample to demonstrate the issue):
public App()
{
BlobCache.ApplicationName = "MyApp";
BlobCache.EnsureInitialized();
// The root page of your application
MainPage = GetMainPage();
}
public object BlockingGetExternalUser()
{
return GetExternalUser().Result;
}
private async Task<object> GetExternalUser()
{
try
{
return await BlobCache.LocalMachine.GetObject<object>("user");
}
catch (KeyNotFoundException)
{
return null;
}
}
The Key "user" does not exist, so I would expect to get a KeyNotFoundException. However I never see this exception being thrown. Instead it just "hangs" and never returns from the await GetObject call.
I am running this on my phone with Android 5.0.
Any ideas how to fix this? Am I doing something fundamentally wrong?
Update: On a side note: Instead of immediately trying GetObject, one could try to check if the key actually exists in the cache and only then retrieve it from the cache. However, if I am not mistaken, there is no other way to do a check other than calling GetObject and catching the exception like in the sample above. For a scenario where one would just want to know if an item exists, that doesn't seem to be ideal. Maybe an "Exists()" method would be a nice to have in Akavache? Or maybe I am missing something?
Update2: Changing the example to not use an async method in the constructor. Just to prove a point that that is not the issue.
Update3: Removing the call from the constructor. When I call BlockingGetExternalUser from anywhere in my code, the await will still hang.

You're most certainly having a dead-lock. Quoting Synchronously waiting for an async operation, and why does Wait() freeze the program here:
The await inside your asynchronous method is trying to come back to the UI thread.
Since the UI thread is busy waiting for the entire task to complete, you have a > deadlock.
Note that your call to .Result implies a Task.Wait() somewhere.
There are two solutions: Either completely avoid the async methods, or wrap your code into a Task.Run like this:
public object BlockingGetExternalUser()
{
return Task.Run<object>(() => GetExternalUser().Result);
}
(I hope it's compiling I didn't verify in VS :)
By experience I tend to avoid async methods in combination with SQLite these days. Reason is that most SQLite wrapper libraries use the Task.Run anti-pattern to provide async wrappers around their methods while SQLite doesn't have any intrinsic notations of being asynchronous. Note though that it's perfectly fine for you to wrap things into Task.Run to make them asynchronous and that it's only an anti-pattern for library designers, suggesting to their users that methods are asynchronous when they're actually not. You can read more about this here: Task.Run as an anti-pattern?

Using async methods in a constructors (var externalUser = GetExternalUser().Result;) is considered as a bad code. You shouldn't use async methods in a class constructors. Read this: Can constructors be async?
You could try to change it to avoid deadlocks:
Func<Task> task = async () => { await GetExternalUser().ConfigureAwait(false); };
task().Wait();
... but I won't recommend it.