We Keep Coding

How to limit number of async IO tasks to database?

I have a list of id's and I want to get data for each of those id in parallel from database. My below ExecuteAsync method is called at very high throughput and for each request we have around 500 ids for which I need to extract data.
So I have got below code where I am looping around list of ids and making async calls for each of those id in parallel and it works fine.
private async Task<List<T>> ExecuteAsync<T>(IList<int> ids, IPollyPolicy policy,
Func<CancellationToken, int, Task<T>> mapper) where T : class
{
var tasks = new List<Task<T>>(ids.Count);
// invoking multiple id in parallel to get data for each id from database
for (int i = 0; i < ids.Count; i++)
{
tasks.Add(Execute(policy, ct => mapper(ct, ids[i])));
}
// wait for all id response to come back
var responses = await Task.WhenAll(tasks);
var excludeNull = new List<T>(ids.Count);
for (int i = 0; i < responses.Length; i++)
{
var response = responses[i];
if (response != null)
{
excludeNull.Add(response);
}
}
return excludeNull;
}
private async Task<T> Execute<T>(IPollyPolicy policy,
Func<CancellationToken, Task<T>> requestExecuter) where T : class
{
var response = await policy.Policy.ExecuteAndCaptureAsync(
ct => requestExecuter(ct), CancellationToken.None);
if (response.Outcome == OutcomeType.Failure)
{
if (response.FinalException != null)
{
// log error
throw response.FinalException;
}
}
return response?.Result;
}
Question:
Now as you can see I am looping all ids and making bunch of async calls to database in parallel for each id which can put lot of load on database (depending on how many request is coming). So I want to limit the number of async calls we are making to database. I modified ExecuteAsync to use Semaphore as shown below but it doesn't look like it does what I want it to do:
private async Task<List<T>> ExecuteAsync<T>(IList<int> ids, IPollyPolicy policy,
Func<CancellationToken, int, Task<T>> mapper) where T : class
{
var throttler = new SemaphoreSlim(250);
var tasks = new List<Task<T>>(ids.Count);
// invoking multiple id in parallel to get data for each id from database
for (int i = 0; i < ids.Count; i++)
{
await throttler.WaitAsync().ConfigureAwait(false);
try
{
tasks.Add(Execute(policy, ct => mapper(ct, ids[i])));
}
finally
{
throttler.Release();
}
}
// wait for all id response to come back
var responses = await Task.WhenAll(tasks);
// same excludeNull code check here
return excludeNull;
}
Does Semaphore works on Threads or Tasks? Reading it here looks like Semaphore is for Threads and SemaphoreSlim is for tasks.
Is this correct? If yes then what's the best way to fix this and limit the number of async IO tasks we make to database here.

Task is an abstraction on threads, and doesn’t necessarily create a new thread. Semaphore limits the number of threads that can access that for loop. Execute returns a Task which aren’t threads. If there’s only 1 request, there will be only 1 thread inside that for loop, even if it is asking for 500 ids. The 1 thread sends off all the async IO tasks itself.
Sort of. I would not say that tasks are related to threads at all. There are actually two kinds of tasks: a delegate task (which is kind of an abstraction of a thread), and a promise task (which has nothing to do with threads).
Regarding the SemaphoreSlim, it does limit the concurrency of a block of code (not threads).
I recently started playing with C# so my understanding is not right looks like w.r.t Threads and Tasks.
I recommend reading my async intro and best practices. Follow up with There Is No Thread if you're interested more about how threads aren't really involved.
I modified ExecuteAsync to use Semaphore as shown below but it doesn't look like it does what I want it to do
The current code is only throttling the adding of the tasks to the list, which is only done one at a time anyway. What you want to do is throttle the execution itself:
private async Task<List<T>> ExecuteAsync<T>(IList<int> ids, IPollyPolicy policy, Func<CancellationToken, int, Task<T>> mapper) where T : class
{
var throttler = new SemaphoreSlim(250);
var tasks = new List<Task<T>>(ids.Count);
// invoking multiple id in parallel to get data for each id from database
for (int i = 0; i < ids.Count; i++)
tasks.Add(ThrottledExecute(ids[i]));
// wait for all id response to come back
var responses = await Task.WhenAll(tasks);
// same excludeNull code check here
return excludeNull;
async Task<T> ThrottledExecute(int id)
{
await throttler.WaitAsync().ConfigureAwait(false);
try {
return await Execute(policy, ct => mapper(ct, id)).ConfigureAwait(false);
} finally {
throttler.Release();
}
}
}

Your colleague has probably in mind the Semaphore class, which is indeed a thread-centric throttler, with no asynchronous capabilities.
Limits the number of threads that can access a resource or pool of resources concurrently.
The SemaphoreSlim class is a lightweight alternative to Semaphore, which includes the asynchronous method WaitAsync, that makes all the difference in the world. The WaitAsync doesn't block a thread, it blocks an asynchronous workflow. Asynchronous workflows are cheap (usually less than 1000 bytes each). You can have millions of them "running" concurrently at any given moment. This is not the case with threads, because of the 1 MB of memory that each thread reserves for its stack.
As for the ExecuteAsync method, here is how you could refactor it by using the LINQ methods Select, Where, ToArray and ToList:
Update: The Polly library supports capturing and continuing on the current synchronization context, so I added a bool executeOnCurrentContext
argument to the API. I also renamed the asynchronous Execute method to ExecuteAsync, to be in par with the guidelines.
private async Task<List<T>> ExecuteAsync<T>(IList<int> ids, IPollyPolicy policy,
Func<CancellationToken, int, Task<T>> mapper,
int concurrencyLevel = 1, bool executeOnCurrentContext = false) where T : class
{
var throttler = new SemaphoreSlim(concurrencyLevel);
Task<T>[] tasks = ids.Select(async id =>
{
await throttler.WaitAsync().ConfigureAwait(executeOnCurrentContext);
try
{
return await ExecuteAsync(policy, ct => mapper(ct, id),
executeOnCurrentContext).ConfigureAwait(false);
}
finally
{
throttler.Release();
}
}).ToArray();
T[] results = await Task.WhenAll(tasks).ConfigureAwait(false);
return results.Where(r => r != null).ToList();
}
private async Task<T> ExecuteAsync<T>(IPollyPolicy policy,
Func<CancellationToken, Task<T>> function,
bool executeOnCurrentContext = false) where T : class
{
var response = await policy.Policy.ExecuteAndCaptureAsync(
ct => executeOnCurrentContext ? function(ct) : Task.Run(() => function(ct)),
CancellationToken.None, continueOnCapturedContext: executeOnCurrentContext)
.ConfigureAwait(executeOnCurrentContext);
if (response.Outcome == OutcomeType.Failure)
{
if (response.FinalException != null)
{
ExceptionDispatchInfo.Throw(response.FinalException);
}
}
return response?.Result;
}

You are throttling the rate at which you add tasks to the list. You are not throttling the rate at which tasks are executed. To do that, you'd probably have to implement your semaphore calls inside the Execute method itself.
If you can't modify Execute, another way to do it is to poll for completed tasks, sort of like this:
for (int i = 0; i < ids.Count; i++)
{
var pendingCount = tasks.Count( t => !t.IsCompleted );
while (pendingCount >= 500) await Task.Yield();
tasks.Add(Execute(policy, ct => mapper(ct, ids[i])));
}
await Task.WhenAll( tasks );

Actually the TPL is capable to control the task execution and limit the concurrency. You can test how many parallel tasks is suitable for your use-case. No need to think about threads, TPL will manage everything fine for you.
To use limited concurrency see this answer, credits to #panagiotis-kanavos
.Net TPL: Limited Concurrency Level Task scheduler with task priority?
The example code is (even using different priorities, you can strip that):
QueuedTaskScheduler qts = new QueuedTaskScheduler(TaskScheduler.Default,4);
TaskScheduler pri0 = qts.ActivateNewQueue(priority: 0);
TaskScheduler pri1 = qts.ActivateNewQueue(priority: 1);
Task.Factory.StartNew(()=>{ },
CancellationToken.None,
TaskCreationOptions.None,
pri0);
Just throw all your tasks to the queue and with Task.WhenAll you can wait till everything is done.

Unexpected behavior with await inside a ContinueWith block

I have a slightly complex requirement of performing some tasks in parallel, and having to wait for some of them to finish before continuing. Now, I am encountering unexpected behavior, when I have a number of tasks, that I want executed in parallel, but inside a ContinueWith handler. I have whipped up a small sample to illustrate the problem:
var task1 = Task.Factory.StartNew(() =>
{
Console.WriteLine("11");
Thread.Sleep(1000);
Console.WriteLine("12");
}).ContinueWith(async t =>
{
Console.WriteLine("13");
var innerTasks = new List<Task>();
for (var i = 0; i < 10; i++)
{
var j = i;
innerTasks.Add(Task.Factory.StartNew(() =>
{
Console.WriteLine("1_" + j + "_1");
Thread.Sleep(500);
Console.WriteLine("1_" + j + "_2");
}));
}
await Task.WhenAll(innerTasks.ToArray());
//Task.WaitAll(innerTasks.ToArray());
Thread.Sleep(1000);
Console.WriteLine("14");
});
var task2 = Task.Factory.StartNew(() =>
{
Console.WriteLine("21");
Thread.Sleep(1000);
Console.WriteLine("22");
}).ContinueWith(t =>
{
Console.WriteLine("23");
Thread.Sleep(1000);
Console.WriteLine("24");
});
Console.WriteLine("1");
await Task.WhenAll(task1, task2);
Console.WriteLine("2");
The basic pattern is:
- Task 1 should be executed in parallel with Task 2.
- Once the first part of part 1 is done, it should do some more things in parallel. I want to complete, once everything is done.
I expect the following result:
1 <- Start
11 / 21 <- The initial task start
12 / 22 <- The initial task end
13 / 23 <- The continuation task start
Some combinations of "1_[0..9]_[1..2]" and 24 <- the "inner" tasks of task 1 + the continuation of task 2 end
14 <- The end of the task 1 continuation
2 <- The end
Instead, what happens, is that the await Task.WhenAll(innerTasks.ToArray()); does not "block" the continuation task from completing. So, the inner tasks execute after the outer await Task.WhenAll(task1, task2); has completed. The result is something like:
1 <- Start
11 / 21 <- The initial task start
12 / 22 <- The initial task end
13 / 23 <- The continuation task start
Some combinations of "1_[0..9]_[1..2]" and 24 <- the "inner" tasks of task 1 + the continuation of task 2 end
2 <- The end
Some more combinations of "1_[0..9]_[1..2]" <- the "inner" tasks of task 1
14 <- The end of the task 1 continuation
If, instead, I use Task.WaitAll(innerTasks.ToArray()), everything seems to work as expected. Of course, I would not want to use WaitAll, so I won't block any threads.
My questions are:
Why is this unexpected behavior occuring?
How can I remedy the situation without blocking any threads?
Thanks a lot in advance for any pointers!

You're using the wrong tools. Instead of StartNew, use Task.Run. Instead of ContinueWith, use await:
var task1 = Task1();
var task2 = Task2();
Console.WriteLine("1");
await Task.WhenAll(task1, task2);
Console.WriteLine("2");
private async Task Task1()
{
await Task.Run(() =>
{
Console.WriteLine("11");
Thread.Sleep(1000);
Console.WriteLine("12");
});
Console.WriteLine("13");
var innerTasks = new List<Task>();
for (var i = 0; i < 10; i++)
{
innerTasks.Add(Task.Run(() =>
{
Console.WriteLine("1_" + i + "_1");
Thread.Sleep(500);
Console.WriteLine("1_" + i + "_2");
}));
await Task.WhenAll(innerTasks);
}
Thread.Sleep(1000);
Console.WriteLine("14");
}
private async Task Task2()
{
await Task.Run(() =>
{
Console.WriteLine("21");
Thread.Sleep(1000);
Console.WriteLine("22");
});
Console.WriteLine("23");
Thread.Sleep(1000);
Console.WriteLine("24");
}
Task.Run and await are superior here because they correct a lot of unexpected behavior in StartNew/ContinueWith. In particular, asynchronous delegates and (for Task.Run) always using the thread pool.
I have more detailed info on my blog regarding why you shouldn't use StartNew and why you shouldn't use ContinueWith.

As noted in the comments, what you're seeing is normal. The Task returned by ContinueWith() completes when the delegate passed to and invoked by ContinueWith() finishes executing. This happens the first time the anonymous method uses the await statement, and the delegate returns a Task object itself that represents the eventual completion of the entire anonymous method.
Since you are only waiting on the ContinueWith() task, and this task only represents the availability of the task that represents the anonymous method, not the completion of that task, your code doesn't wait.
From your example, it's not clear what the best fix is. But if you make this small change, it will do what you want:
await Task.WhenAll(await task1, task2);
I.e. in the WhenAll() call, don't wait on the ContinueWith() task itself, but rather on the task that task will eventually return. Use await here to avoid blocking the thread while you wait for that task to be available.

When using async methods/lambdas with StartNew, you either wait on the returned task and the contained task:
var task = Task.Factory.StartNew(async () => { /* ... */ });
task.Wait();
task.Result.Wait();
// consume task.Result.Result
Or you use the extension method Unwrap on the result of StartNew and wait on the task it returns.
var task = Task.Factory.StartNew(async () => { /* ... */ })
.Unwrap();
task.Wait();
// consume task.Result
The following discussion goes along the line that Task.Factory.StartNew and ContinueWith should be avoided in specific cases, such as when you don't provide creation or continuation options or when you don't provide a task scheduler.
I don't agree that Task.Factory.StartNew shouldn't be used, I agree that you should use (or consider using) Task.Run wherever you use a Task.Factory.StartNew method overload that doesn't take TaskCreationOptions or a TaskScheduler.
Note that this only applies to the default Task.Factory. I've used custom task factories where I chose to use the StartNew overloads without options and task scheduler, because I configured the factories specific defaults for my needs.
Likewise, I don't agree that ContinueWith shouldn't be used, I agree that you should use (or consider using) async/await wherever you use a ContinueWith method overload that doesn't take TaskContinuationOptions or a TaskScheduler.
For instance, up to C# 5, the most practical way to workaround the limitation of await not being supported in catch and finally blocks is to use ContinueWith.
C# 6:
try
{
return await something;
}
catch (SpecificException ex)
{
await somethingElse;
// throw;
}
finally
{
await cleanup;
}
Equivalent before C# 6:
return await something
.ContinueWith(async somethingTask =>
{
var ex = somethingTask.Exception.InnerException as SpecificException;
if (ex != null)
{
await somethingElse;
// await somethingTask;
}
},
CancellationToken.None,
TaskContinuationOptions.DenyChildAttach | TaskContinuationOptions.NotOnRanToCompletion,
TaskScheduler.Default)
.Unwrap()
.ContinueWith(async catchTask =>
{
await cleanup;
await catchTask;
},
CancellationToken.None,
TaskContinuationOptions.DenyChildAttach,
TaskScheduler.Default)
.Unwrap();
Since, as I told, in some cases I have a TaskFactory with specific defaults, I've defined a few extension methods that take a TaskFactory, reducing the error chance of not passing one of the arguments (I known I can always forget to pass the factory itself):
public static Task ContinueWhen(this TaskFactory taskFactory, Task task, Action<Task> continuationAction)
{
return task.ContinueWith(continuationAction, taskFactory.CancellationToken, taskFactory.ContinuationOptions, taskFactory.Scheduler);
}
public static Task<TResult> ContinueWhen<TResult>(this TaskFactory taskFactory, Task task, Func<Task, TResult> continuationFunction)
{
return task.ContinueWith(continuationFunction, taskFactory.CancellationToken, taskFactory.ContinuationOptions, taskFactory.Scheduler);
}
// Repeat with argument combinations:
// - Task<TResult> task (instead of non-generic Task task)
// - object state
// - bool notOnRanToCompletion (useful in C# before 6)
Usage:
// using namespace that contains static task extensions class
var task = taskFactory.ContinueWhen(existsingTask, t => Continue(a, b, c));
var asyncTask = taskFactory.ContinueWhen(existingTask, async t => await ContinueAsync(a, b, c))
.Unwrap();
I decided not to mimic Task.Run by not overloading the same method name to unwrapping task-returning delegates, it's really not always what you want. Actually, I didn't even implement ContinueWhenAsync extension methods so you need to use Unwrap or two awaits.
Often, these continuations are I/O asynchronous operations, and the pre- and post-processing overhead should be so small that you shouldn't care if it starts running synchronously up to the first yielding point, or even if it completes synchronously (e.g. using an underlying MemoryStream or a mocked DB access). Also, most of them don't depend on a synchronization context.
Whenever you apply the Unwrap extension method or two awaits, you should check if the task falls in this category. If so, async/await is most probably a better choice than starting a task.
For asynchronous operations with a non-negligible synchronous overhead, starting a new task may be preferable. Even so, a notable exception where async/await is still a better choice is if your code is async from the start, such as an async method invoked by a framework or host (ASP.NET, WCF, NServiceBus 6+, etc.), as the overhead is your actual business. For long processing, you may consider using Task.Yield with care. One of the tenets of asynchronous code is to not be too fine grained, however, too coarse grained is just as bad: a set of heavy-duty tasks may prevent the processing of queued lightweight tasks.
If the asynchronous operation depends on a synchronization context, you can still use async/await if you're within that context (in this case, think twice or more before using .ConfigureAwait(false)), otherwise, start a new task using a task scheduler from the respective synchronization context.

Prevent overlapping async code execution

I have read about async-await patterns and that Mutex is incompatible with asynchronous code so I wonder how to write the following method in a lightweight way, without object allocations (not even for a Task<>) but without excessive encumberance (from a pattern).
The problem is as follows. Look at this method:
public async void SendAsync(Stream stream, byte[] data)
{
await stream.WriteAsync(data.Length);
await stream.WriteAsync(data);
await stream.FlushAsync();
}
Now, this method probably is not very useful. In fact there is plenty more code, but for the problem at hand it's perfectly complete.
The problem here is that I have a single main thread that will invoke the method very quickly without await:
for (i = 0; i < 10; i++)
{
SendAsync(stream[i], buffer[i]);
}
All I want is that the main thread loops through quickly without blocking.
Now, this loop is looped over, too:
while (true)
{
for (i = 0; i < 10; i++)
{
SendAsync(stream[i], buffer[i]);
}
// TODO: add a break criterion here
}
My suspicion (fear) is that a very chunky buffer (say 1 MB) on a congested stream[i] (yes, it's a TCP/IP NetworkStream underneath) may cause the nth invocation to occur before the preceding n-1th invocation was reentrant (returned from the invocation).
The two tasks will then interfere with each other.
I don't want any of these two solutions I've already discarded:
(Non-)Solution 1: Mutex
This will throw an exception because Mutex has thread affinity and async code is thread-agnostic:
public async void SendAsync(Stream stream, byte[] data)
{
mutex.WaitOne(); // EXCEPTION!
await stream.WriteAsync(data.Length);
await stream.WriteAsync(data);
await stream.FlushAsync();
mutex.ReleaseMutex();
}
Solution 2: return a Task to *.WaitAll()
The loop is in consumer land while SendAsync() is in framework land, I don't want to force consumers to use an intricate pattern:
while (true)
{
for (i = 0; i < 10; i++)
{
tasks[i] = SendAsync(stream[i], buffer[i]);
}
Task.WaitAll(tasks);
// TODO: add a break criterion here
}
Furthermore I don't ABSOLUTELY want Tasks to be allocated on each loop, this code will run at a moderate frequency (5-10 Hz) with 1000+ open streams. I cannot allocate 10000+ objects per second, it would GC like crazy. Mind that this is minimal code per SO policy, the actual code is much more complex.
What I'd like to see is some kind of outside-loop-allocation that allows to wait for completion only when absolutely necessary (like a Mutex) but at the same time does not allocate memory inside the loop.

Let's say you have a wrapper class over the stream, you could do something like this :
private Stream stream;
private Task currentTask;
public async void SendAsync(byte[] data)
{
currentTask = WriteAsyncWhenStreamAvailable(data);
await currentTask;
}
private async Task WriteAsyncWhenStreamAvailable(byte[] data)
{
if (currentTask != null)
await currentTask.ConfigureAwait(false);
await WriteAsync(data);
}
public async Task WriteAsync(byte[] data)
{
await stream.WriteAsync(data.Length).ConfigureAwait(false);
await stream.WriteAsync(data).ConfigureAwait(false);
await stream.FlushAsync().ConfigureAwait(false);
}
This way you always wait for the previous WriteAsync to end before sending the new data. This solution will ensure the buffers to be sent in requested order.
You could use an async SemaphoreSlim instead of a Mutex
private SemaphoreSlim semaphore = new SemaphoreSlim(1);
public async void SendAsync(Stream stream, byte[] data)
{
await semaphore.WaitAsync(); // EXCEPTION!
await stream.WriteAsync(data.Length);
await stream.WriteAsync(data);
await stream.FlushAsync();
semaphore.Release();
}
Beware of OutOfMemoryException if you blindlessly produce new data without taking in account the size of pending buffers...

Inside a loop,does each async call get chained to the returned task using task's continuewith?

The best practice is to collect all the async calls in a collection inside the loop and do Task.WhenAll(). Yet, want to understand what happens when an await is encountered inside the loop, what would the returned Task contain? what about further async calls? Will it create new tasks and add them to the already returned Task sequentially?
As per the code below
private void CallLoopAsync()
{
var loopReturnedTask = LoopAsync();
}
private async Task LoopAsync()
{
int count = 0;
while(count < 5)
{
await SomeNetworkCallAsync();
count++;
}
}
The steps I assumed are
LoopAsync gets called
count is set to zero, code enters while loop, condition is checked
SomeNetworkCallAsync is called,and the returned task is awaited
New task/awaitable is created
New task is returned to CallLoopAsync()
Now, provided there is enough time for the process to live, How / In what way, will the next code lines like count++ and further SomeNetworkCallAsync be executed?
Update - Based on Jon Hanna and Stephen Cleary:
So there is one Task and the implementation of that Task will involve
5 calls to NetworkCallAsync, but the use of a state-machine means
those tasks need not be explicitly chained for this to work. This, for
example, allows it to decide whether to break the looping or not based
on the result of a task, and so on.
Though they are not chained, each call will wait for the previous call to complete as we have used await (in state m/c, awaiter.GetResult();). It behaves as if five consecutive calls have been made and they are executed one after the another (only after the previous call gets completed). If this is true, we have to be bit more careful in how we are composing the async calls.For ex:
Instead of writing
private async Task SomeWorkAsync()
{
await SomeIndependentNetworkCall();// 2 sec to complete
var result1 = await GetDataFromNetworkCallAsync(); // 2 sec to complete
await PostDataToNetworkAsync(result1); // 2 sec to complete
}
It should be written
private Task[] RefactoredSomeWorkAsync()
{
var task1 = SomeIndependentNetworkCall();// 2 sec to complete
var task2 = GetDataFromNetworkCallAsync()
.ContinueWith(result1 => PostDataToNetworkAsync(result1)).Unwrap();// 4 sec to complete
return new[] { task1, task2 };
}
So that we can say RefactoredSomeWorkAsync is faster by 2 seconds, because of the possibility of parallelism
private async Task CallRefactoredSomeWorkAsync()
{
await Task.WhenAll(RefactoredSomeWorkAsync());//Faster, 4 sec
await SomeWorkAsync(); // Slower, 6 sec
}
Is this correct? - Yes. Along with "async all the way", "Accumulate tasks all the way" is good practice. Similar discussion is here

When count is zero, new task will be created because of await and be returned
No. It will not. It will simply call the async method consequently, without storing or returning the result. The value in loopReturnedTask will store the Task of LoopAsync, not related to SomeNetworkCallAsync.
await SomeNetworkCallAsync(); // call, wait and forget the result
You may want to read the MSDN article on async\await.

To produce code similar to what async and await do, if those keywords didn't exist, would require code a bit like:
private struct LoopAsyncStateMachine : IAsyncStateMachine
{
public int _state;
public AsyncTaskMethodBuilder _builder;
public TestAsync _this;
public int _count;
private TaskAwaiter _awaiter;
void IAsyncStateMachine.MoveNext()
{
try
{
if (_state != 0)
{
_count = 0;
goto afterSetup;
}
TaskAwaiter awaiter = _awaiter;
_awaiter = default(TaskAwaiter);
_state = -1;
loopBack:
awaiter.GetResult();
awaiter = default(TaskAwaiter);
_count++;
afterSetup:
if (_count < 5)
{
awaiter = _this.SomeNetworkCallAsync().GetAwaiter();
if (!awaiter.IsCompleted)
{
_state = 0;
_awaiter = awaiter;
_builder.AwaitUnsafeOnCompleted<TaskAwaiter, TestAsync.LoopAsyncStateMachine>(ref awaiter, ref this);
return;
}
goto loopBack;
}
_state = -2;
_builder.SetResult();
}
catch (Exception exception)
{
_state = -2;
_builder.SetException(exception);
return;
}
}
[DebuggerHidden]
void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine param0)
{
_builder.SetStateMachine(param0);
}
}
public Task LoopAsync()
{
LoopAsyncStateMachine stateMachine = new LoopAsyncStateMachine();
stateMachine._this = this;
AsyncTaskMethodBuilder builder = AsyncTaskMethodBuilder.Create();
stateMachine._builder = builder;
stateMachine._state = -1;
builder.Start(ref stateMachine);
return builder.Task;
}
(The above is based on what happens when you use async and await except that the result of that uses names that cannot be valid C# class or field names, along with some extra attributes. If its MoveNext() reminds you of an IEnumerator that's not entirely irrelevant, the mechanism by which await and async produce an IAsyncStateMachine to implement a Task is similar in many ways to how yield produces an IEnumerator<T>).
The result is a single Task which comes from AsyncTaskMethodBuilder and makes use of LoopAsyncStateMachine (which is close to the hidden struct that the async produces). Its MoveNext() method is first called upon the task being started. It will then use an awaiter on SomeNetworkCallAsync. If it is already completed it moves on to the next stage (increment count and so on), otherwise it stores the awaiter in a field. On subsequent uses it will be called because the SomeNetworkCallAsync() task has returned, and it will get the result (which is void in this case, but could be a value if values were returned). It then attempts further loops and again returns when it is waiting on a task that is not yet completed.
When it finally reaches a count of 5 it calls SetResult() on the builder, which sets the result of the Task that LoopAsync had returned.
So there is one Task and the implementation of that Task will involve 5 calls to NetworkCallAsync, but the use of a state-machine means those tasks need not be explicitly chained for this to work. This, for example, allows it to decide whether to break the looping or not based on the result of a task, and so on.

When an async method first yields at an await, it returns a Task (or Task<T>). This is not the task being observed by the await; it is a completely different task created by the async method. The async state machine controls the lifetime of that Task.
One way to think of it is to consider the returned Task as representing the method itself. The returned Task will only complete when the method completes. If the method returns a value, then that value is set as the result of the task. If the method throws an exception, then that exception is captured by the state machine and placed on that task.
So, there's no need for attaching continuations to the returned task. The returned task will not complete until the method is done.
How / In what way, will the next code lines like count++ and further SomeNetworkCallAsync be executed?
I do explain this in my async intro post. In summary, when a method awaits, it captures a "current context" (SynchronizationContext.Current unless it is null, in which case it uses TaskScheduler.Current). When the await completes, it resumes executing its async method within that context.
That's what technically happens; but in the vast majority of cases, this simply means:
If an async method starts on a UI thread, then it will resume on that same UI thread.
If an async method starts within an ASP.NET request context, then it will resume with that same request context (not necessarily on the same thread, though).
Otherwise, the async method resumes on a thread pool thread.

async all the way down issue

I have an async asp.net controller. This controller calls an async method. The method that actually performs the async IO work is deep down in my application. The series of methods between the controller and the last method in the chain are all marked with the async modifier. Here is an example of how I have the code setup:
public async Task<ActionResult> Index(int[] ids)
{
List<int> listOfDataPoints = dataPointService(ids);
List<Task> dpTaskList = new List<Task>();
foreach (var x in listOfDataPoints)
{
dpTaskList.Add(C_Async(x));
}
await Task.WhenAll(dpTaskList);
return View();
}
private async Task C_Async(int id)
{
//this method executes very fast
var idTemp = paddID(id);
await D_Async(idTemp);
}
private async Task D_Async(string id)
{
//this method executes very fast
await E_Async(id);
}
private async Task E_Async(string url)
{
//this method performs the actual async IO
result = await new WebClient().DownloadStringTaskAsync(new Uri(url))
saveContent(result);
}
As you can see the controller calls C_Async(x) asynchronously then there is a chain of async methods to E_Async. There are methods between the controller and E_Async and all have the async modifier. Is there a performance penalty since there are methods using the async modifyer but not doing any async IO work?
Note: This is a simplified version of the real code there are more async methods between the controller and the E_Async method.

Yes. There is a penalty (though not a huge one), and if you don't need to be async don't be. This pattern is often called "return await" where you can almost always remove both the async and the await. Simply return the task you already have that represents the asynchronous operations:
private Task C_Async(int id)
{
// This method executes very fast
var idTemp = paddID(id);
return D_Async(idTemp);
}
private Task D_Async(string id)
{
// This method executes very fast
return E_Async(id);
}
In this specific case Index will only await the tasks that E_Async returns. That means that after all the I/O is done the next line of code will directly be return View();. C_Async and D_Async already ran and finished in the synchronous call.

You must be careful about the thread message pumps and what async really does. The sample below calls into an async method which calls two other async methods which start two tasks to do the actual work which wait 2 and 3 seconds.
13.00 6520 .ctor Calling async method
13.00 6520 RunSomethingAsync Before
13.00 6520 GetSlowString Before
13.00 5628 OtherTask Sleeping for 2s
15.00 5628 OtherTask Sleeping done
15.00 6520 GetVerySlow Inside
15.00 2176 GetVerySlow Sleeping 3s
18.00 2176 GetVerySlow Sleeping Done
18.00 6520 RunSomethingAsync After GetSlowOtherTaskResultGetVerySlowReturn
As you can see the calls are serialized which might not be what you want when you after performance. Perhaps the two distinct await calls do not depend on each other and can be started directly as tasks.
All methods until GetSlowStringBefore are called on the UI or ASP.NET thread that started the async operation (if it it has a message pump). Only the last call with the result of the operation are marshalled back to the initiating thread.
The performance penalty is somewhere in the ContextSwitch region to wake up an already existing thread. This should be somewhere at microsecond level. The most expensive stuff would be the creation of the managed objects and the garbage collector cleaning up the temporary objects. If you call this in a tight loop you will be GC bound because there is an upper limit how many threads can be created. In that case TPL will buffer your tasks in queues which require memory allocations and then drain the queues with n worker threads from the thread pool.
On my Core I7 I get an overhead of 2microseconds for each call (comment out the Debug.Print line) and a memory consumption of 6,5GB for 5 million calls in a WPF application which gives you a memory overhead of 130KB per asynchronous operation chain. If you are after high scalability you need to watch after your GC. Until Joe Duffy has finished his new language we have to use CLR we currently have.
public partial class MainWindow : Window
{
public MainWindow()
{
InitializeComponent();
Print("Calling async method");
RunSomethingAsync();
}
private async void RunSomethingAsync()
{
Print("Before");
string msg = await GetSlowString();
Print("After " + msg);
cLabel.Content = msg;
}
void Print(string message, [CallerMemberName] string method = "")
{
Debug.Print("{0:N2} {1} {2} {3}", DateTime.Now.Second, AppDomain.GetCurrentThreadId(), method, message);
}
private async Task<string> GetSlowString()
{
Print("Before");
string otherResult = await OtherTask();
return "GetSlow" + otherResult + await GetVerySlow(); ;
}
private Task<string> OtherTask()
{
return Task.Run(() =>
{
Print("Sleeping for 2s");
Thread.Sleep(2 * 1000);
Print("Sleeping done");
return "OtherTaskResult";
});
}
private Task<string> GetVerySlow()
{
Print("Inside");
return Task.Run(() =>
{
Print("Sleeping 3s");
Thread.Sleep(3000);
Print("Sleeping Done");
return "GetVerySlowReturn";
});
}
}