We have an asp.net application, it makes no use of async or parallel methods(except for some owin middleware).
I want to start using the async version of some methods using the async-await syntax.
I know that there is danger in mixing async and sync operations as that might cause a deadlocks.
It is not possible to rewrite the whole app in one go.
Where should I Start? is the safe way to do it to make the controllers actions async and work my way down? or the other way around?
Is there some clear warning sign(s) that I can watch for along the way like: "never have a sync method call an async method, but it's fine to have async call sync"
Where should I Start? is the safe way to do it to make the controllers actions async and work my way down? or the other way around?
I recommend starting at the "leaves" and creating "vertical partitions". That is, within a service or data access layer, identify I/O-bound operations. Then create asynchronous copies of those methods that use naturally-asynchronous APIs. Once you have those, then continue "up" your layers, creating asynchronous copies, until you can make a controller action asynchronous.
You'll end up with a fair amount of code duplication - both synchronous and asynchronous versions of the same method - but this is only temporary. Once all the other code only uses the asynchronous version of a method, the synchronous version can be removed.
You can safely mix sync and async IO. The deadlock arises from the way await automatically uses the ambient SynchronizationContext. Platform methods such as Stream.Read almost never do this.
Start with making long-running IO async. Short-running IO is more likely not to benefit.
It's completely safe to mix async and sync operations in your code. What's dangerous is using "sync over async" when you block on asynchronous operations inside synchronous ones.
As long as when you change a single operation you make sure it's async all the way you can add more async operations gradually.
Related
I have started refactoring old code to become Async. If the method is not straight forward to async then i'm copying the old method and I create a new MethodAsync(), and then i just upgrade one by one client to use the new async method when I can.
But I would instead like to avoid creating this duplicate methods. So i'm wondering if there is performance issues with the following two scenarios:
UPDATE: Ignore point 1
Method is async with await keyword(s). But the client is still not async so it is using .Result :- forcing it to sync behavior. Is this way of "forcing" sync behavior more bad than calling a method that does not use async keyword?
Update: The Async State machine created with the async keyword would introduce overhead. So another question would be how much impact does that have, keep in mind this project is huge in the numbers of several thousand methods.
And 4:- Are there any other factors than the creation of Async state machine that comes to play with the performance while the methods are not yet true async?
But the client is still not async so it is using .Result :- forcing it to sync behavior. Is this way of "forcing" sync behavior more bad than calling a method that does not use async keyword?
It's generally considered an antipattern. The main reason is that it can cause deadlocks, depending on the client.
Update: The Async State machine created with the async keyword would introduce overhead. So another question would be how much impact does that have, keep in mind this project is huge in the numbers of several thousand methods.
Any overhead from the async keyword is minimal, compared to the overhead of blocking threads on asynchronous code.
I would instead like to avoid creating this duplicate methods.
I recommend reading my article on Brownfield Async Development; in particular, the "boolean argument hack" is one I've used with some success in this scenario.
Is this way of "forcing" sync behavior more bad than calling a method that does not use async keyword?
As mentioned by #mong Zhu, this risks causing deadlocks. Say that the method is awaiting some work that needs to be done on the main thread, calling .Result on the mainthread will prevent the main-thread from being used, effectivly deadlocking the application.
The Async State machine created with the async keyword would introduce overhead. So another question would be how much impact does that have
This is almost impossible to answer since performance spans multiple orders of magnitude. The overhead should be small when doing things Async/await is intended for, i.e. IO operations or compute bound operations sufficiently slow for the user to notice. The general recommendation is as always: measure.
Are there any other factors than the creation of Async state machine that comes to play with the performance while the methods are not yet true async
The state machine is simply a hidden class with a large switch statement. I have not measured this, but I suspect the overhead of the state machine is smaller than scheduling work on another thread. However, marking a method as async is pointless unless you are not using await. If you are returning tasks but not using await you can either:
Return the task returned from another method. Say that you do some initial work and call ReadToEndAsync that returns a Task<string>, just return the task as is.
use Task.FromResult(myObject) to create a completed task, or Task.CompletedTask;
Edit: Return a ValueTask instead, this is either a task or a TResult. Since this is a struct there is very little overhead to create one.
In this article: https://blog.stephencleary.com/2013/11/taskrun-etiquette-examples-dont-use.html , it is advised against using Task.Run. however there are lot of libraries that provide methods that ends with Async and hence I expect those methods to return a running task that I can await (which however is not necessary, since those libraries could decide to return a synchronous task).
The context is a ASP.NET application. How am I supposed to make a method running in parallel?
What I understand is that async calls are executed in parallel if they contain at least one "await" operator inside, the problem is that the innermost call, should be parallel to achieve that, and to do that I have somewhat to resort to Task.Run
I have also seen some examples using TaskCompletionSource, is this necessary to implement the "inner most async method" to run a method in parallel in a ASP.NET application?
In an ASP.Net application we tend to value requests/s over individual response times1 - certainly if we're directly trading off one versus the other. So we don't try to focus more CPU power at satisfying one request.
And really, focussing more CPU power at a task is what Task.Run is for - it's for when you have a distinct chunk of work to be done, you can't do it on the current thread (because its got its own work to do) and when you're free to use as much CPU as possible.
In ASP.Net, where async shines is when we're dealing with I/O. Nasty slow things like accessing the file system or talking to a database across the network. And wonderfully, at the lowest level, the windows I/O system is async already and we don't have to devote a thread just to waiting for things to finish.
So, you won't be using Task.Run. Instead you'll be looking for I/O related objects that expose Async methods. And those methods themselves will not, as above, be using Task.Run. What this does allow us to do is to stop using any threads for servicing our particular request whilst there's no work to be done, and so improve out requests/s metric.
1This is a generalization but single user/request ASP.Net sites are rare in my experience.
I have been applying async best practices to all my libraries. Basically it means:
Only use async when it's truly async (libraries shouldn't lie)
Define a synchronous method if and only if you have a faster synchronous method that won’t dead lock.
Postfix all async methods with Async
I worked on a library that is synchronous by nature. This means it has only sync methods. If the user wants to run the work on a separate thread than the UI thread, they can do that themselves by using Task.Factory (responsibility of the caller).
However, inside a handler / method / extensibility point, we want to show the user a message box. This is an async method (for example, WinRT ShowDialogAsync). Then this gives us the following options:
A. Move everything to async (so we have the option to use await in our handlers and don't block anything).
public async Task MyMethodAsync()
{
await _messageService.ShowAsync();
}
The advantage is that users can add async methods without having to use .Wait(). The downside is that we are lying as a library (it's not truly async).
I have considered making everything async, but I don't think that's a good idea either. It would make all libraries lie but prepare them in case we would need it. Remember that making everything async out of the box has a (small) performance impact as well.
B. Inside the handler that requires user input, call .Wait()
public void MyMethod()
{
_messageService.ShowAsync().Wait();
}
The advantage is that this will allow us to use async code inside sync methods. But... it will never be callable from the UI-thread because the _messageService dispatches to the UI thread (but it cannot do that because it's still waiting for the method, resulting in a deadlock). This method will work when used inside a Task.Factory.Run block (but the responsibility is up to the end-user):
await Task.Factory.Run(() => MyMethod());
The question
I feel that both have pros and cons, but what would you choose? Let the library lie (A) or only allow the method to be called from a background thread (B)? Or maybe there are other options I've overseen.
If I go for A, it means I have to bump the major version every time (because it's actually a breaking change) whenever a user requests to convert a method to an async signature method.
Define a synchronous method if and only if you have a faster synchronous method that won’t dead lock.
I'd say "define a synchronous method if you have synchronous work to do". It doesn't matter how fast it is. The burden is on the caller to determine if it's too slow and they need to use Task.Run.
However, inside a handler / method / extensibility point
If this is an Observer kind of extensibility, consider just using events or observables.
However, it sounds like you want more of a Strategy kind of extensibility, where your invoking code must wait for and/or change its behavior based on the result of the callback.
I have considered making everything async, but I don't think that's a good idea either.
Async all the way is a guideline, not a strict command. It definitely applies in the 99% case, but this could be one of the exceptions. I would try not to make a library async just for the sake of a possibly-async Strategy pattern; I'd investigate other extension possibilities first. There is a valid argument for making the library async, if you view the Strategy callback as a dependency (the library would be async because its dependency is (possibly) async).
As you've discovered, there's no clean way to do sync-over-async. There are a few different hacks (such as blocking from a background thread), but you'll first need to decide whether you need to call your library from the UI thread.
If you do, then there's just two options: make the library async, or use a nested message loop. I strongly avoid nested message loops, especially in libraries; I'm just mentioning it for sake of completeness.
If you can impose on the user a requirement to only call the library from a non-UI thread, then you can apply other hacks. E.g., blocking the background thread.
There's not an easy solution, sorry.
As far as me personally... if the library needs an async Strategy, then I would lean towards making the library async. But it does depend on what kind of library it is, whether there were backwards-compatibility issues, etc. And the first thing I'd look into is a different kind of extensibility point.
as you can read here :
https://msdn.microsoft.com/en-us/magazine/jj991977.aspx
Async All the Way
Asynchronous code reminds me of the story of a fellow who mentioned that the world was suspended in space and was immediately challenged by an elderly lady claiming that the world rested on the back of a giant turtle. When the man enquired what the turtle was standing on, the lady replied, “You’re very clever, young man, but it’s turtles all the way down!” As you convert synchronous code to asynchronous code, you’ll find that it works best if asynchronous code calls and is called by other asynchronous code—all the way down (or “up,” if you prefer). Others have also noticed the spreading behavior of asynchronous programming and have called it “contagious” or compared it to a zombie virus. Whether turtles or zombies, it’s definitely true that asynchronous code tends to drive surrounding code to also be asynchronous. This behavior is inherent in all types of asynchronous programming, not just the new async/await keywords.
“Async all the way” means that you shouldn’t mix synchronous and asynchronous code without carefully considering the consequences. In particular, it’s usually a bad idea to block on async code by calling Task.Wait or Task.Result. This is an especially common problem for programmers who are “dipping their toes” into asynchronous programming, converting just a small part of their application and wrapping it in a synchronous API so the rest of the application is isolated from the changes. Unfortunately, they run into problems with deadlocks. After answering many async-related questions on the MSDN forums, Stack Overflow and e-mail, I can say this is by far the most-asked question by async newcomers once they learn the basics: “Why does my partially async code deadlock?”
Probably this question has already been made, but I never found a definitive answer. Let's say that I have a Web API 2.0 Application hosted on IIS. I think I understand that best practice (to prevent deadlocks on client) is always use async methods from the GUI event to the HttpClient calls. And this is good and it works. But what is the best practice in case I had client application that does not have a GUI (e.g. Window Service, Console Application) but only synchronous methods from which to make the call? In this case, I use the following logic:
void MySyncMethodOnMyWindowServiceApp()
{
list = GetDataAsync().Result().ToObject<List<MyClass>>();
}
async Task<Jarray> GetDataAsync()
{
list = await Client.GetAsync(<...>).ConfigureAwait(false);
return await response.Content.ReadAsAsync<JArray>().ConfigureAwait(false);
}
But unfortunately this can still cause deadlocks on client that occur at random times on random machines.
The client app stops at this point and never returns:
list = await Client.GetAsync(<...>).ConfigureAwait(false);
If it's something that can be run in the background and isn't forced to be synchronous, try wrapping the code (that calls the async method) in a Task.Run(). I'm not sure that'll solve a "deadlock" problem (if it's something out of sync, that's another issue), but if you want to benefit from async/await, if you don't have async all the way down, I'm not sure there's a benefit unless you run it in a background thread. I had a case where adding Task.Run() in a few places (in my case, from an MVC controller which I changed to be async) and calling async methods not only improved performance slightly, but it improved reliability (not sure that it was a "deadlock" but seemed like something similar) under heavier load.
You will find that using Task.Run() is regarded by some as a bad way to do it, but I really couldn't see a better way to do it in my situation, and it really did seem to be an improvement. Perhaps this is one of those things where there's the ideal way to do it vs. the way to make it work in the imperfect situation that you're in. :-)
[Updated due to requests for code]
So, as someone else posted, you should do "async all the way down". In my case, my data wasn't async, but my UI was. So, I went async down as far as I could, then I wrapped my data calls with Task.Run in such as way that it made sense. That's the trick, I think, to figure out if it makes sense that things can run in parallel, otherwise, you're just being synchronous (if you use async and immediately resolve it, forcing it to wait for the answer). I had a number of reads that I could perform in parallel.
In the above example, I think you have to async up as far as makes sense, and then at some point, determine where you can spin off a t hread and perform the operation independent of the other code. Let's say you have an operation that saves data, but you don't really need to wait for a response -- you're saving it and you're done. The only thing you might have to watch out for is not to close the program without waiting for that thread/task to finish. Where it makes sense in your code is up to you.
Syntax is pretty easy. I took existing code, changed the controller to an async returning a Task of my class that was formerly being returned.
var myTask = Task.Run(() =>
{
//...some code that can run independently.... In my case, loading data
});
// ...other code that can run at the same time as the above....
await Task.WhenAll(myTask, otherTask);
//..or...
await myTask;
//At this point, the result is available from the task
myDataValue = myTask.Result;
See MSDN for probably better examples:
https://msdn.microsoft.com/en-us/library/hh195051(v=vs.110).aspx
[Update 2, more relevant for the original question]
Let's say that your data read is an async method.
private async Task<MyClass> Read()
You can call it, save the task, and await on it when ready:
var runTask = Read();
//... do other code that can run in parallel
await runTask;
So, for this purpose, calling async code, which is what the original poster is requesting, I don't think you need Task.Run(), although I don't think you can use "await" unless you're an async method -- you'll need an alternate syntax for Wait.
The trick is that without having some code to run in parallel, there's little point in it, so thinking about multi-threading is still the point.
Using Task<T>.Result is the equivalent of Wait which will perform a synchronous block on the thread. Having async methods on the WebApi and then having all the callers synchronously blocking them effectively makes the WebApi method synchronous. Under load you will deadlock if the number of simultaneous Waits exceeds the server/app thread pool.
So remember the rule of thumb "async all the way down". You want the long running task (getting a collection of List) to be async. If the calling method must be sync you want to make that conversion from async to sync (using either Result or Wait) as close to the "ground" as possible. Keep they long running process async and have the sync portion as short as possible. That will greatly reduce the length of time that threads are blocked.
So for example you can do something like this.
void MySyncMethodOnMyWindowServiceApp()
{
List<MyClass> myClasses = GetMyClassCollectionAsync().Result;
}
Task<List<MyClass>> GetMyListCollectionAsync()
{
var data = await GetDataAsync(); // <- long running call to remote WebApi?
return data.ToObject<List<MyClass>>();
}
The key part is the long running task remains async and not blocked because await is used.
Also don't confuse the responsiveness with scalability. Both are valid reasons for async. Yes responsiveness is a reason for using async (to avoid blocking on the UI thread). You are correct this wouldn't apply to a back end service however this isn't why async is used on a WebApi. The WebApi is also a non GUI back end process. If the only advantage of async code was responsiveness of the UI layer then WebApi would be sync code from start to finish. The other reason for using async is scalability (avoiding deadlocks) and this is the reason why WebApi calls are plumbed async. Keeping the long running processes async helps IIS make more efficient use of a limited number of threads. By default there are only 12 worker threads per core. This can be raised but that isn't a magic bullet either as threads are relatively expensive (about 1MB overhead per thread). await allows you to do more with less. More concurrent long running processes on less threads before a deadlock occurs.
The problem you are having with deadlocks must stem from something else. Your use of ConfigureAwait(false) prevents deadlocks here. Solve the bug and you are fine.
See Should we switch to use async I/O by default? to which the answer is "no". You should decide on a case by case basis and choose async when the benefits outweigh the costs. It is important to understand that async IO has a productivity cost associated with it. In non-GUI scenarios only a few targeted scenarios derive any benefit at all from async IO. The benefits can be enormous, though, but only in those cases.
Here's another helpful post: https://stackoverflow.com/a/25087273/122718
I have some concerns using async actions in ASP.NET MVC. When does it improve performance of my apps, and when does it not?
Is it good to use async action everywhere in ASP.NET MVC?
Regarding awaitable methods: shall I use async/await keywords when I want to query a database (via EF/NHibernate/other ORM)?
How many times can I use await keywords to query the database asynchronously in one single action method?
You may find my MSDN article on the subject helpful; I took a lot of space in that article describing when you should use async on ASP.NET, not just how to use async on ASP.NET.
I have some concerns using async actions in ASP.NET MVC. When it improves performance of my apps, and when - not.
First, understand that async/await is all about freeing up threads. On GUI applications, it's mainly about freeing up the GUI thread so the user experience is better. On server applications (including ASP.NET MVC), it's mainly about freeing up the request thread so the server can scale.
In particular, it won't:
Make your individual requests complete faster. In fact, they will complete (just a teensy bit) slower.
Return to the caller/browser when you hit an await. await only "yields" to the ASP.NET thread pool, not to the browser.
First question is - is it good to use async action everywhere in ASP.NET MVC?
I'd say it's good to use it everywhere you're doing I/O. It may not necessarily be beneficial, though (see below).
However, it's bad to use it for CPU-bound methods. Sometimes devs think they can get the benefits of async by just calling Task.Run in their controllers, and this is a horrible idea. Because that code ends up freeing up the request thread by taking up another thread, so there's no benefit at all (and in fact, they're taking the penalty of extra thread switches)!
Shall I use async/await keywords when I want to query database (via EF/NHibernate/other ORM)?
You could use whatever awaitable methods you have available. Right now most of the major players support async, but there are a few that don't. If your ORM doesn't support async, then don't try to wrap it in Task.Run or anything like that (see above).
Note that I said "you could use". If you're talking about ASP.NET MVC with a single database backend, then you're (almost certainly) not going to get any scalability benefit from async. This is because IIS can handle far more concurrent requests than a single instance of SQL server (or other classic RDBMS). However, if your backend is more modern - a SQL server cluster, Azure SQL, NoSQL, etc - and your backend can scale, and your scalability bottleneck is IIS, then you can get a scalability benefit from async.
Third question - How many times I can use await keywords to query database asynchronously in ONE single action method?
As many as you like. However, note that many ORMs have a one-operation-per-connection rule. In particular, EF only allows a single operation per DbContext; this is true whether the operation is synchronous or asynchronous.
Also, keep in mind the scalability of your backend again. If you're hitting a single instance of SQL Server, and your IIS is already capable of keeping SQLServer at full capacity, then doubling or tripling the pressure on SQLServer is not going to help you at all.
Asynchronous action methods are useful when an action must perform several independent long running operations.
A typical use for the AsyncController class is long-running Web
service calls.
Should my database calls be asynchronous ?
The IIS thread pool can often handle many more simultaneous blocking requests than a database server. If the database is the bottleneck, asynchronous calls will not speed up the database response. Without a throttling mechanism, efficiently dispatching more work to an overwhelmed database server by using asynchronous calls merely shifts more of the burden to the database. If your DB is the bottleneck, asynchronous calls won’t be the magic bullet.
You should have a look at 1 and 2 references
Derived from #PanagiotisKanavos comments:
Moreover, async doesn't mean parallel. Asynchronous execution frees a
valuable threadpool thread from blocking for an external resource, for
no complexity or performance cost. This means the same IIS machine can
handle more concurrent requests, not that it will run faster.
You should also consider that blocking calls start with a
CPU-intensive spinwait. During stress times, blocking calls will
result in escalating delays and app pool recycling. Asynchronous calls
simply avoid this
is it good to use async action everywhere in ASP.NET MVC?
As usual in programming, it depends. There is always a trade-off when going down a certain path.
async-await shines in places where you know you'll receiving concurrent requests to your service and you want to be able to scale out well. How does async-await help with scaling out? In the fact that when you invoke a async IO call synchronously, such as a network call or hitting your database, the current thread which is responsible for the execution is blocked waiting for the request to finish. When you use async-await, you enable the framework to create a state machine for you which makes sure that after the IO call is complete, your method continues executing from where it left off.
A thing to note is that this state machine has a subtle overhead. Making a method asynchronous does not make it execute faster, and that is an important factor to understand and a misconception many people have.
Another thing to take under consideration when using async-await is the fact that it is async all the way, meaning that you'll see async penetrate your entire call stack, top to buttom. This means that if you want to expose synchronous API's, you'll often find yourself duplicating a certain amount of code, as async and sync don't mix very well.
Shall I use async/await keywords when I want to query database (via
EF/NHibernate/other ORM)?
If you choose to go down the path of using async IO calls, then yes, async-await will be a good choice, as more and more modern database providers expose async method implementing the TAP (Task Asynchronous Pattern).
How many times I can use await keywords to query database
asynchronously in ONE single action method?
As many as you want, as long as you follow the rules stated by your database provider. There is no limit to the amount of async calls you can make. If you have queries which are independent of each other and can be made concurrently, you can spin a new task for each and use await Task.WhenAll to wait for both to complete.
async actions help best when the actions does some I\O operations to DB or some network bound calls where the thread that processes the request will be stalled before it gets answer from the DB or network bound call which you just invoked. It's best you use await with them and it will really improve the responsiveness of your application (because less ASP input\output threads will be stalled while waiting for the DB or any other operation like that). In all my applications whenever many calls to DB very necessary I've always wrapped them in awaiatable method and called that with await keyword.
My 5 cents:
Use async/await if and only if you do an IO operation, like DB or external service webservice.
Always prefer async calls to DB.
Each time you query the DB.
P.S. There are exceptional cases for point 1, but you need to have a good understanding of async internals for this.
As an additional advantage, you can do few IO calls in parallel if needed:
Task task1 = FooAsync(); // launch it, but don't wait for result
Task task2 = BarAsync(); // launch bar; now both foo and bar are running
await Task.WhenAll(task1, task2); // this is better in regard to exception handling
// use task1.Result, task2.Result
As you know, MVC supports asynchronous controllers and you should take advantage of it. In case your Controller, performs a lengthy operation, (it might be a disk based I/o or a network call to another remote service), if the request is handled in synchronous manner, the IIS thread is busy the whole time. As a result, the thread is just waiting for the lengthy operation to complete. It can be better utilized by serving other requests while the operation requested in first is under progress. This will help in serving more concurrent requests.
Your webservice will be highly scalable and will not easily run into C10k problem.
It is a good idea to use async/await for db queries. and yes you can use them as many number of times as you deem fit.
Take a look here for excellent advise.
My experience is that today a lot of developers use async/await as a default for controllers.
My suggestion would be, use it only when you know it will help you.
The reason is, as Stephen Cleary and others already mentioned, it can introduce performance issues, rather than resolving them, and it will help you only in a specific scenario:
High-traffic controllers
Scalable backend
Is it good to use async action everywhere in ASP.NET MVC?
It's good to do so wherever you can use an async method especially when you have performance issues at the worker process level which happens for massive data and calculation operations. Otherwise, no need because unit testing will need casting.
Regarding awaitable methods: shall I use async/await keywords when I
want to query a database (via EF/NHibernate/other ORM)?
Yes, it's better to use async for any DB operation as could as possible to avoid performance issues at the level of worker processes.
Note that EF has created many async alternatives for most operations, such as:
.ToListAsync()
.FirstOrDefaultAsync()
.SaveChangesAsync()
.FindAsync()
How many times can I use await keywords to query the database
asynchronously in one single action method?
The sky is the limit