Background
We are currently developing a web application, which relies on ASP .NET MVC 5, Angular.JS 1.4, Web API 2 and Entity Framework 6. For scalability reasons, the web application heavility relies on the async/await pattern. Our domain requires some cpu-intensive calculations, which can takes some seconds (<10s). In the past some team members used Task.Run, in order to speed up the calculations.Since starting an extra thread inside ASP .NET MVC or Web API controllers is considered a bad practise (the thread is not known by the IIS, so not considered on AppDomain Recycle => See Stephen Cleary's blog post), they used ConfigureAwait(false).
Example
public async Task CalculateAsync(double param1, double param2)
{
// CalculateSync is synchronous and cpu-intensive (<10s)
await Task.Run(() => this.CalculateSync(param1, param2))).ConfigureAwait(false);
}
Questions
Is there any performance benefit in using Task.Run in an async Web API Controller for cpu-bound operations?
Does ConfigureAwait(false) really avoid the creation of an extra thread?
Is there any performance benefit in using Task.Run in an async Web API Controller for cpu-bound operations?
Zero. None. In fact, you're hindering performance by spawning a new thread. Within the context of a web application, spawning a thread is not the same thing as running in the "background". This is due to the nature of a web request. When there's an incoming request, a thread is taken from the pool to service the request. Using async allows the thread to be returned before the end of the request, if and only if the thread is in a wait-state, i.e. idle. Spawning a thread to do work on, effectively idles the primary thread, allowing it to be returned to the pool, but you've still got an active thread. Returning the original thread to the pool does nothing at that point. Then, when the new thread finishes its work, you've got to request a main thread back from the pool, and finally return the response. The response cannot be returned until all work has completed, so whether you use 1 thread or a hundred, async or sync, the response cannot be returned until everything finishes. Therefore, using additional threads does nothing but add overhead.
Does ConfigureAwait(false) really avoid the creation of an extra thread?
No, or more appropriately, it's not about that. ConfigureAwait is just an optimization hint, and only determines whether the original context is maintained between thread jumps. Long and short, it has nothing to do with the creation of a thread, and at least in the context of an ASP.NET application, has negligible performance impact either way.
Is there any performance benefit in using Task.Run in an async Web API Controller for cpu-bound operations?
No. And it doesn't matter whether it's CPU bound or not.
Task.Run offloads work to a ThreadPool thread. The web api request already uses a ThreadPool thread so you're just limiting scalability by offloading to another thread with no reason.
This is useful in UI apps, where the UI thread is a special single thread.
Does ConfigureAwait(false) really avoid the creation of an extra thread?
It doesn't affect thread creating in one way or another. All it does is configures whether to resume on the captured SynchronizationContext or not.
Is there any performance benefit in using Task.Run in an async Web API Controller for cpu-bound operations?
Think about what really happens - Task.Run() creates a Task on the thread pool, and your await operator will free the thread (I'm assuming all methods down the stack are also awaiting). Now your thread is back to the pool, and it might pick up that same Task! In this scenario, there is obviously no performance gain. There is performance hit, actually. But if the thread picks up another Task (that's probably what will happen), another thread would have to pick up CalculateSync() Task and continue from where the former stopped. It would have made more sense to let the original thread execute CalculateSync() in the first place, no Tasks involved, and let the other thread have the other queued Tasks.
Does ConfigureAwait(false) really avoid the creation of an extra thread?
Not at all. It merely points out that the continuation shouldn't be executed on the caller's context.
There is one more thing that you need to consider. As you told your your api is doing CPU intensive task then async/await help to run the process in another thread and free your app pool for another request. Means your web api can handle more number of request per second if you use async/await correctly.
In your case look like this.CalculateSync(param1, param2) is non-async method so to call this asynchronously you have to use Task.Run.
I'll recommend to remove .ConfigureAwait(false) as this will actually decrease your performance.
Related
So Task.Yield will yield execution back to the caller immediately, essentially making whatever is bellow it in a continuation of the task returned by it (Task.Yield).
Assuming an Asp.Net Core application which does not have a SynchronizationContext, is there any functional difference from awaiting Task.Yield and passing that continuation to Task.Run?
Assuming an Asp.Net Core application which does not have a SynchronizationContext, is there any functional difference from awaiting Task.Yield and passing that continuation to Task.Run?
There's practically no difference. await Task.Yield will queue the continuation to the thread pool and return to the caller. await Task.Run will queue its delegate to the thread pool and return to the caller.
Personally, I prefer Task.Run because it has a very explicit intent and because it's unambiguous (i.e., in other scenarios such as GUI threads, await Task.Yield will not queue to the thread pool, while await Task.Run always does the same thing regardless of context).
As others have noted in the comments, there's no reason to actually do either one on ASP.NET Core, which is already running on thread pool threads and does not have a request context. It would add a little overhead to the request while providing no benefit at all.
They're difference, if you take the result as educational view point. Better explaination can be found here, here and... if you want to see how it compile to IL, take a shot here.
From real world project, as i saw the only purpose and benefit from Task.Yield is to force a task to run asynchronous. (decorate a method as async/await doesn't ensure that they will run asynchronously, especially with the new ValueTask).
So, forcing immediate return the execution to the caller, means that you doesn't care it's final result (for example: you call an API, but never care about the response).
Which according to me, in the asp.net core server side process, have a better approach, that's build a FireAndForget service and pass the task to it, let they run on a separate scope would be much safer.
That's cause the Task.Yield will capture the current context and goes on with it. So an exception would very likely to happen if you use resource that related to the current execution scope, HttpRequest is a particular case, which normally finish in less than a second(HttpContext already got disposed).
I have a question regarding some code I am writing. I have 3 calls made synchronously to some endpoints that have large payloads. I don’t want to wait for these payloads and instead continue running through the method until I need the values from those 3 endpoints.
I have approached a solution like this. I converted the method that calls the 3 service endpoints into an async method. I start the call for the data using
var serviceCallOneTask = Task.Run(()=> serviceCallOne());
Note serviceCallOne() is not asynchronous
and finally when I need the data I do something like
var serviceCallOneValue = await serviceCallOneTask;
My questions are
Is this solution considered bad practice?
Should I be worried about deadlocks?
From what I have read, when using await we are not blocking a thread but when using task.run we are using a CPU-bound thread and we are blocking the thread pool; is that correct?
Is it better for me to convert everything in this httpGet method from beginning to end into async methods?
Is it ok for me to approach the problem this way for now and later on convert those task.run() services into asynchronous methods?
Is this solution considered bad practice?
It depends what you're doing.
Task.Run moves execution to another thread. That is helpful in a desktop application because you don't want long-running CPU-bound operations running on the UI thread and locking up your UI.
ASP.NET is different since there is no UI thread, so there is no need to move operations to another thread, unless you want to do something in parallel (run two CPU-bound operations at the same time).
If you're doing something else between calling Task.Run and await serviceCallOneTask, then that's certainly a reason to do what you're doing. But whether it's "better" depends on what serviceCallOne() is doing. You have to think about two things to determine if the benefit outweighs the cost:
Does the benefit of running it in a separate thread outweigh the cost of moving it to a separate thread? (Is it actually faster than running it in the same thread?)
Remember that ASP.NET has a limited number of threads (by default, 20 per processor), and now you're using 2 threads instead of 1. Depending on the expected load of your application, that may or may not matter.
Should I be worried about deadlocks?
Not with the small bit of code that you've shown. As long as you don't wait synchronously on an async method, you will not have to worry about deadlocks.
From what I have read, when using await we are not blocking a thread but when using task.run we are using a CPU-bound thread and we are blocking the thread pool; is that correct?
When using await, you don't block the current thread. As discussed above, you might just be blocking another thread, depending on your code.
Is it better for me to convert everything in this httpGet method from beginning to end into async methods?
Considering the limited thread that ASP.NET has, and that async/await helps you free up threads, then yes. It's always better to use async wherever you can.
Is it ok for me to approach the problem this way for now and later on convert those task.run() services into asynchronous methods?
If it works, then it's "ok". But you have to change something, right? May as well do it right. :)
I'm trying to benchmark (using Apache bench) a couple of ASP.NET Web API 2.0 endpoints. One of which is synchronous and one async.
[Route("user/{userId}/feeds")]
[HttpGet]
public IEnumerable<NewsFeedItem> GetNewsFeedItemsForUser(string userId)
{
return _newsFeedService.GetNewsFeedItemsForUser(userId);
}
[Route("user/{userId}/feeds/async")]
[HttpGet]
public async Task<IEnumerable<NewsFeedItem>> GetNewsFeedItemsForUserAsync(string userId)
{
return await Task.Run(() => _newsFeedService.GetNewsFeedItemsForUser(userId));
}
After watching Steve Sanderson's presentation I issued the following command ab -n 100 -c 10 http://localhost.... to each endpoint.
I was surprised as the benchmarks for each endpoint seemed to be approximately the same.
Going off what Steve explained I was expecting that the async endpoint would be more performant because it would release thread pool threads back to the thread pool immediately, thus making them available for other requests and improving throughput. But the numbers seem exactly the same.
What am I misunderstanding here?
Using await Task.Run to create "async" WebApi is a bad idea - you will still use a thread, and even from the same thread pool used for requests.
It will lead to some unpleasant moments described in good details here:
Extra (unnecessary) thread switching to the Task.Run thread pool thread. Similarly, when that thread finishes the request, it has to
enter the request context (which is not an actual thread switch but
does have overhead).
Extra (unnecessary) garbage is created. Asynchronous programming is a tradeoff: you get increased responsiveness at the expense of higher
memory usage. In this case, you end up creating more garbage for the
asynchronous operations that is totally unnecessary.
The ASP.NET thread pool heuristics are thrown off by Task.Run “unexpectedly” borrowing a thread pool thread. I don’t have a lot of
experience here, but my gut instinct tells me that the heuristics
should recover well if the unexpected task is really short and would
not handle it as elegantly if the unexpected task lasts more than two
seconds.
ASP.NET is not able to terminate the request early, i.e., if the client disconnects or the request times out. In the synchronous case,
ASP.NET knew the request thread and could abort it. In the
asynchronous case, ASP.NET is not aware that the secondary thread pool
thread is “for” that request. It is possible to fix this by using
cancellation tokens, but that’s outside the scope of this blog post.
Basically, you do not allow any asynchrony to the ASP.NET - you just hide the CPU-bound synchronous code behind the async facade. Async on its own is ideal for I/O bound code, because it allows to utilize CPU (threads) at their top efficiency (no blocking for I/O), but when you have Compute-bound code, you will still have to utilize CPU to the same extent.
And taking into account the additional overhead from Task and context switching you will get even worser results than with simple sync controller methods.
HOW TO MAKE IT TRULY ASYNC:
GetNewsFeedItemsForUser method shall be turned into async.
[Route("user/{userId}/feeds/async")]
[HttpGet]
public async Task<IEnumerable<NewsFeedItem>> GetNewsFeedItemsForUserAsync(string userId)
{
return await _newsFeedService.GetNewsFeedItemsForUser(userId);
}
To do it:
If it is some library method then look for its async variant (if there are none - bad luck, you'll have to search for some competing analogue).
If it is your custom method using file system or database then leverage their async facilities to create async API for the method.
When you have server-side code (i.e. some ApiController) and your functions are asynchronous - so they return Task<SomeObject> - is it considered best practice that any time you await functions that you call ConfigureAwait(false)?
I had read that it is more performant since it doesn't have to switch thread contexts back to the original thread context. However, with ASP.NET Web Api, if your request is coming in on one thread, and you await some function and call ConfigureAwait(false) that could potentially put you on a different thread when you are returning the final result of your ApiController function.
I've typed up an example of what I am talking about below:
public class CustomerController : ApiController
{
public async Task<Customer> Get(int id)
{
// you are on a particular thread here
var customer = await GetCustomerAsync(id).ConfigureAwait(false);
// now you are on a different thread! will that cause problems?
return customer;
}
}
Update: ASP.NET Core does not have a SynchronizationContext. If you are on ASP.NET Core, it does not matter whether you use ConfigureAwait(false) or not.
For ASP.NET "Full" or "Classic" or whatever, the rest of this answer still applies.
Original post (for non-Core ASP.NET):
This video by the ASP.NET team has the best information on using async on ASP.NET.
I had read that it is more performant since it doesn't have to switch thread contexts back to the original thread context.
This is true with UI applications, where there is only one UI thread that you have to "sync" back to.
In ASP.NET, the situation is a bit more complex. When an async method resumes execution, it grabs a thread from the ASP.NET thread pool. If you disable the context capture using ConfigureAwait(false), then the thread just continues executing the method directly. If you do not disable the context capture, then the thread will re-enter the request context and then continue to execute the method.
So ConfigureAwait(false) does not save you a thread jump in ASP.NET; it does save you the re-entering of the request context, but this is normally very fast. ConfigureAwait(false) could be useful if you're trying to do a small amount of parallel processing of a request, but really TPL is a better fit for most of those scenarios.
However, with ASP.NET Web Api, if your request is coming in on one thread, and you await some function and call ConfigureAwait(false) that could potentially put you on a different thread when you are returning the final result of your ApiController function.
Actually, just doing an await can do that. Once your async method hits an await, the method is blocked but the thread returns to the thread pool. When the method is ready to continue, any thread is snatched from the thread pool and used to resume the method.
The only difference ConfigureAwait makes in ASP.NET is whether that thread enters the request context when resuming the method.
I have more background information in my MSDN article on SynchronizationContext and my async intro blog post.
Brief answer to your question: No. You shouldn't call ConfigureAwait(false) at the application level like that.
TL;DR version of the long answer: If you are writing a library where you don't know your consumer and don't need a synchronization context (which you shouldn't in a library I believe), you should always use ConfigureAwait(false). Otherwise, the consumers of your library may face deadlocks by consuming your asynchronous methods in a blocking fashion. This depends on the situation.
Here is a bit more detailed explanation on the importance of ConfigureAwait method (a quote from my blog post):
When you are awaiting on a method with await keyword, compiler
generates bunch of code in behalf of you. One of the purposes of this
action is to handle synchronization with the UI (or main) thread. The key
component of this feature is the SynchronizationContext.Current which
gets the synchronization context for the current thread.
SynchronizationContext.Current is populated depending on the
environment you are in. The GetAwaiter method of Task looks up for
SynchronizationContext.Current. If current synchronization context is
not null, the continuation that gets passed to that awaiter will get
posted back to that synchronization context.
When consuming a method, which uses the new asynchronous language
features, in a blocking fashion, you will end up with a deadlock if
you have an available SynchronizationContext. When you are consuming
such methods in a blocking fashion (waiting on the Task with Wait
method or taking the result directly from the Result property of the
Task), you will block the main thread at the same time. When
eventually the Task completes inside that method in the threadpool, it
is going to invoke the continuation to post back to the main thread
because SynchronizationContext.Current is available and captured. But
there is a problem here: the UI thread is blocked and you have a
deadlock!
Also, here are two great articles for you which are exactly for your question:
The Perfect Recipe to Shoot Yourself in The Foot - Ending up with a Deadlock Using the C# 5.0 Asynchronous Language Features
Asynchronous .NET Client Libraries for Your HTTP API and Awareness of async/await's Bad Effects
Finally, there is a great short video from Lucian Wischik exactly on this topic: Async library methods should consider using Task.ConfigureAwait(false).
The biggest draw back I've found with using ConfigureAwait(false) is that the thread culture is reverted to the system default. If you've configured a culture e.g ...
<system.web>
<globalization culture="en-AU" uiCulture="en-AU" />
...
and you're hosting on a server whose culture is set to en-US, then you will find before ConfigureAwait(false) is called CultureInfo.CurrentCulture will return en-AU and after you will get en-US.
i.e.
// CultureInfo.CurrentCulture ~ {en-AU}
await xxxx.ConfigureAwait(false);
// CultureInfo.CurrentCulture ~ {en-US}
If your application is doing anything which requires culture specific formatting of data, then you'll need to be mindful of this when using ConfigureAwait(false).
I have some general thoughts about the implementation of Task:
Task is disposable yet we are not supposed to use using.
ConfigureAwait was introduced in 4.5. Task was introduced in 4.0.
.NET Threads always used to flow the context (see C# via CLR book) but in the default implementation of Task.ContinueWith they do not b/c it was realised context switch is expensive and it is turned off by default.
The problem is a library developer should not care whether its clients need context flow or not hence it should not decide whether flow the context or not.
[Added later] The fact that there is no authoritative answer and proper reference and we keep fighting on this means someone has not done their job right.
I have got a few posts on the subject but my take - in addition to Tugberk's nice answer - is that you should turn all APIs asynchronous and ideally flow the context . Since you are doing async, you can simply use continuations instead of waiting so no deadlock will be cause since no wait is done in the library and you keep the flowing so the context is preserved (such as HttpContext).
Problem is when a library exposes a synchronous API but uses another asynchronous API - hence you need to use Wait()/Result in your code.
I've been reading some async articles here: http://www.asp.net/web-forms/tutorials/aspnet-45/using-asynchronous-methods-in-aspnet-45 and the author says :
When you’re doing asynchronous work, you’re not always using a thread.
For example, when you make an asynchronous web service request,
ASP.NET will not be using any threads between the async method call
and the await.
So what I am trying to understand is, how does it become async if we don't use any Threads for concurrent execution? What does it mean "you're not always using a thread."?
Let me first explain what I know regarding working with threads (A quick example, of course Threads can be used in different situations other than UI and Worker methodology here)
You have UI Thread to take input, give output.
You can handle things in UI Thread but it makes the UI unresponsive.
So lets say we have a stream-related operation and we need to download some sort of data.
And we also allow users to do other things while it is being downloaded.
We create a new worker thread which downloads the file and changes the progress bar.
Once it is done, there is nothing to do so thread is killed.
We continue from UI thread.
We can either wait for the worker thread in UI thread depending on the situation but before that while the file is being downloaded, we can do other things with UI thread and then wait for the worker thread.
Isn't the same for async programming? If not, what's the difference? I read that async programming uses ThreadPool to pull threads from though.
Threads are not necessary for asynchronous programming.
"Asynchronous" means that the API doesn't block the calling thread. It does not mean that there is another thread that is blocking.
First, consider your UI example, this time using actual asynchronous APIs:
You have UI Thread to take input, give output.
You can handle things in UI Thread but it makes the UI unresponsive.
So lets say we have a stream-related operation and we need to download some sort of data.
And we also allow users to do other things while it is being downloaded.
We use asynchronous APIs to download the file. No worker thread is necessary.
The asynchronous operation reports its progress back to the UI thread (which updates the progress bar), and it also reports its completion to the UI thread (which can respond to it like any other event).
This shows how there can be only one thread involved (the UI thread), yet also have asynchronous operations going on. You can start up multiple asynchronous operations and yet only have one thread involved in those operations - no threads are blocked on them.
async/await provides a very nice syntax for starting an asynchronous operation and then returning, and having the rest of the method continue when that operation completes.
ASP.NET is similar, except it doesn't have a main/UI thread. Instead, it has a "request context" for every incomplete request. ASP.NET threads come from a thread pool, and they enter the "request context" when they work on a request; when they're done, they exit their "request context" and return to the thread pool.
ASP.NET keeps track of incomplete asynchronous operations for each request, so when a thread returns to the thread pool, it checks to see if there are any asynchronous operations in progress for that request; if there are none, then the request is complete.
So, when you await an incomplete asynchronous operation in ASP.NET, the thread will increment that counter and return. ASP.NET knows the request isn't complete because the counter is non-zero, so it doesn't finish the response. The thread returns to the thread pool, and at that point: there are no threads working on that request.
When the asynchronous operation completes, it schedules the remainder of the async method to the request context. ASP.NET grabs one of its handler threads (which may or may not be the same thread that executed the earlier part of the async method), the counter is decremented, and the thread executes the async method.
ASP.NET vNext is slightly different; there's more support for asynchronous handlers throughout the framework. But the general concept is the same.
For more information:
My async/await intro post tries to be both an intro yet also reasonably complete picture of how async and await work.
The official async/await FAQ has lots of great links that go into a lot of detail.
The MSDN magazine article It's All About the SynchronizationContext exposes some of the plumbing underneath.
First time when I saw async and await, I thougth they were C# Syntactic sugar for Asynchronous Programming Model. I was wrong, async and await are more than that. It is a brand new asynchronous pattern Task-based Asynchronous Pattern, http://www.microsoft.com/en-us/download/details.aspx?id=19957 is a good article to get start. Most of the FCL classes which inplement TAP are call APM methods (BegingXXX() and EndXXX()). Here are two code snaps for TAP and AMP:
TAP sample:
static void Main(string[] args)
{
GetResponse();
Console.ReadLine();
}
private static async Task<WebResponse> GetResponse()
{
var webRequest = WebRequest.Create("http://www.google.com");
Task<WebResponse> response = webRequest.GetResponseAsync();
Console.WriteLine(new StreamReader(response.Result.GetResponseStream()).ReadToEnd());
return response.Result;
}
APM sample:
static void Main(string[] args)
{
var webRequest = WebRequest.Create("http://www.google.com");
webRequest.BeginGetResponse(EndResponse, webRequest);
Console.ReadLine();
}
static void EndResponse(IAsyncResult result)
{
var webRequest = (WebRequest) result.AsyncState;
var response = webRequest.EndGetResponse(result);
Console.WriteLine(new StreamReader(response.GetResponseStream()).ReadToEnd());
}
Finally these two will be the same, because GetResponseAsync() call BeginGetResponse() and EndGetResponse() inside. When we reflector the source code of GetResponseAsync(), we will get code like this:
task = Task<WebResponse>.Factory.FromAsync(
new Func<AsyncCallback, object, IAsyncResult>(this.BeginGetResponse),
new Func<IAsyncResult, WebResponse>(this.EndGetResponse), null);
For APM, in the BeginXXX(), there is an argument for a callback method which will invoked when the task (typically is an IO heavy operation) was completed. Creating a new thread and asynchronous, both of them will immediately return in main thread, both of them are unblocked. On performance side, creating new thread will cost more resource when process I/O-bound operations such us read file, database operation and network read. There are two disadvantages in creating new thread,
like in your mentioned article, there are memory cost and CLR are
limitation on thread pool.
Context switch will happen. On the other hander, asynchronous will
not create any thread manually and it will not have context switch
when the the IO-bound operations return.
Here is an picture which can help to understand the differences:
This diagram is from a MSDN article "Asynchronous Pages in ASP.NET 2.0", which explain very detail about how the old asynchronous working in ASP.NET 2.0.
About Asynchronous Programming Model, please get more detail from Jeffrey Richter's article "Implementing the CLR Asynchronous Programming Model", also there are more detail on his book "CLR via Csharp 3rd Edition" in chapter 27.
Let’s imagine that you are implementing a web application and as each client request comes in to
your server, you need to make a database request. When a client request comes in, a thread pool
thread will call into your code. If you now issue a database request synchronously, the thread will block
for an indefinite amount of time waiting for the database to respond with the result. If during this time
another client request comes in, the thread pool will have to create another thread and again this
thread will block when it makes another database request. As more and more client requests come in,
more and more threads are created, and all these threads block waiting for the database to respond.
The result is that your web server is allocating lots of system resources (threads and their memory) that
are barely even used!
And to make matters worse, when the database does reply with the various results, threads become
unblocked and they all start executing. But since you might have lots of threads running and relatively
few CPU cores, Windows has to perform frequent context switches, which hurts performance even
more. This is no way to implement a scalable application.
To read data from the file, I now call ReadAsync instead of Read. ReadAsync internally allocates a
Task object to represent the pending completion of the read operation. Then, ReadAsync
calls Win32’s ReadFile function (#1). ReadFile allocates its IRP, initializes it just like it did in the
synchronous scenario (#2), and then passes it down to the Windows kernel (#3). Windows adds the IRP
to the hard disk driver’s IRP queue (#4), but now, instead of blocking your thread, your thread is
allowed to return to your code; your thread immediately returns from its call to ReadAsync (#5, #6,
and #7). Now, of course, the IRP has not necessarily been processed yet, so you cannot have code after
ReadAsync that attempts to access the bytes in the passed-in Byte[].