We Keep Coding

Understanding Async/await properly. How is it correct?

Imagine the following situation. There is an UI and s long running operation must be called without blocking the Main thread. the long running operation calls intself some other methods, which don't interact with the UI thread.
In Most cases, the methods which are called from Method B and C have synchronous alternatives to their Async counterparts. The question is: can they be used safely instead of their async counterpart? Lets say DbContext.DbSet.Add instead of AddAsync
// UI
public async Task UiMethodAsync()
{
var result = await MethodAAsync();
}
// some component
public async Task<bool> MethodAAsync()
{
return await MethodBAsync().ConfigureAwait(false);
}
public async Task<bool> MethodBAsync()
{
return await MethodCAsync().ConfigureAwait(false);
}
public async Task<bool> MethodCAsync()
{
return await DbContext.Set<TEntit>.AnyAsync().ConfigureAwait(false);
}
My question is: Is it neccessary to make all the methods asynchronous to prevent UI thread from blocking or would it be eben good enough to make Method B and C synchronously like this:
// UI
public async Task UiMethodAsync()
{
var result = await MethodAAsync();
}
// some component
public Task<bool> MethodAAsync()
{
return Task.FromResult(MethodB());
}
public bool MethodB()
{
return MethodC();
}
public bool MethodC()
{
return DbContext.Set<TEntit>.Any();
}
of course this depends on what MethodB and C are doing, if they interact with the ui thread or not. but let's say they don't and just have to calculate things and return a result.
Is there a need to make them asynchronous as well? I guess no. I think it probably avoids unnecessary overhead for managing tasks and threads.

A method that returns a Task creates the expectation that will not block the calling thread. At least not for a substantial amount of time. But this is not enforced by the async-await machinery. It is possible, and actually very easy, to write a method that breaks this expectation. For example:
public Task DoStuffTheWrongWayAsync()
{
Thread.Sleep(1000); // Simulate a heavy computation, or a blocking call
return Task.CompletedTask;
}
Any thread that calls this method will be blocked for one second, and then will be handed a completed task. I don't know if this antipattern has an established name. Fake-asynchrony comes in mind, although this can also be used for the case that the caller is not blocked, but another poor thread is blocked instead. The bottom line is that a well behaved asynchronous method should return a Task immediately, leaving the calling thread free to do other work (like responding to UI events if it's a UI thread).

Just adding the async keyword is not enough to make your code not block.
A function returning a Task will still block unless the calls are async all the way down.
It's difficult to explain, but to highlight one bit of your code:
public async Task<bool> MethodA()
{
return Task.FromResult(MethodB());
}
Is equivalent to
public async Task<bool> MethodA()
{
bool b = MethodB();
return Task.FromResult(b);
}
It is clear now that the first line of code is blocking, and the Task isn't created until the second line.

Generally speaking, if you want to make something asynchronous, you start at the lowest level - the APIs that are actually doing I/O work. In your example, AnyAsync would be the first thing made asynchronous. Then allow the asynchrony to grow from there (up through MethodC, then MethodB, then MethodA, and finally UiMethod).
It is normal to end up with asynchrony going "all the way".
Is there a need to make them asynchronous as well? I guess no.
Yes. If you want them to be asynchronous, then they need to be asynchronous. Task.FromResult is for synchronous implementations; they won't be asynchronous.
Imagine the following situation. There is an UI... the methods which are called from Method B and C have synchronous alternatives to their Async counterparts. The question is: can they be used safely instead of their async counterpart?
One thing you can get away with in UI world is by wrapping calls to synchronous methods in Task.Run. E.g.:
public async Task UiMethodAsync()
{
var result = await Task.Run(MethodA);
}
This is a useful technique for situations where you have a client-side app, don't want to block the UI, but also don't want to take the time to convert all the code to being asynchronous. Note that this is not appropriate for server-side applications such as ASP.NET.

How to convert synchronous method to asynchronous without changing it's signature

I have a large scale C# solution with 40-ish modules.
I'm trying to convert a service used solution-wide from synchronous to asynchronous.
the problem is I can't find a way to do so without changing the signature of the method.
I've tried wrapping said asynchronous desired operation with Task but that requires changing the method signature.
I've tried changing the caller to block itself while the method is operating but that screwed my system pretty good because it's a very long calling-chain and changing each of the members in the chain to block itself is a serious issue.
public SomeClass Foo()
{
// Synchronous Code
}
Turn this into:
public SomeClass Foo()
{
//Asynchronous code
}
whilst all callers stay the same
public void DifferentModule()
{
var class = Foo();
}

Any implementation that fundamentally changes something from sync to async is going to involve a signature change. Any other approach is simply not going to work well. Fundamentally: async and sync demand different APIs, which means: different signatures. This is unavoidable, and frankly "How to convert synchronous method to asynchronous without changing it's signature?" is an unsolvable problem (and more probably: the wrong question). I'm sorry if it seems like I'm not answering the question there, but... sometimes the answer is "you can't, and anyone who says you can is tempting you down a very bad path".
In the async/Task<T> sense, the most common way to do this without breaking compatibility is to add a new / separate method, so you have
SomeReturnType Foo();
and
Task<SomeReturnType> FooAsync(); // or ValueTask<T> if often actually synchoronous
nothing that Foo and FooAsync here probably have similar but different implementations - one designed to exploit async, one that works purely synchronously. It is not a good idea to spoof one by calling the other - both "sync over async" (the synchronous version calling the async version) and "async over sync" (the async version calling the sync version) are anti-patterns, and should be avoided (the first is much more harmful than the second).
If you really don't want to do this, you could also do things like adding a FooCompleted callback event (or similar), but : this is still fundamentally a signature change, and the caller will still have to use the API differently. By the time you've done that - you might as well have made it easy for the consumer by adding the Task<T> API instead.

The common pattern is to add an Async to the method name and wrap the return type in a Task. So:
public SomeClass Foo()
{
// Synchronous Code
}
becomes:
public Task<SomeClass> FooAsync()
{
// Asynchronous Code
}
You'll end up with two versions of the method, but it will allow you to gradually migrate your code over to the async approach, and it won't break the existing code whilst you're doing the migration.

If you desperately need to do this, it can be achieved by wrapping the Synchronous code that needs to become Asynchronous in a Task this can be done like this:
public SomeClass Foo()
{
Task t = Task.Run(() =>
{
// Do stuff, code in here will run asynchronously
}
t.Wait();
// or if you need a return value: var result = t.Wait();
return someClass;
// or return result
}
Code you write inside the Task.Run(() => ...) will run asynchronously
Short explanation: with Task t = Task.Run(() => ...) we start a new Task, the "weird" parameter is a Lambda expression, basically we're passing a anonymous Method into the Run method which will get executed by the Task
We then wait for the task to finish with t.Wait();. The Wait method can return a value, you can return a value from an anonymous method just like from any method, with the return keyword
Note: This can, but should not be done. See Sean's answer for more

Understanding how to use Async without Multithreading

I've been reading about asynchronous methods, specifically in C# with the new async/await keywords, and despite much reading and perusing this forum, I still am convinced that async requires multithreading. Please explain what I am misunderstanding!
I understand that you can write an async method without spawning a background thread. Super basic example:
async System.Threading.Tasks.Task<int> GetZeroAsync()
{
return 0;
}
But of course, this method is completely useless to be marked as async because, well, it isn't asynchronous. I also get a compiler warning about the method lacking an "await" operator, as expected. Okay, so what can we await? I can await something like Task.Run(), but that defeats the point, because I'm now using multithreading. Any other example I've found online tries to prove that you don't need multithreading by simply doing something like so:
async System.Threading.Tasks.Task<int> MyMethodAsync()
{
return await CallAnotherAsyncMethod();
}
Maybe I'm misunderstanding this, but all it proves to me is that I'm not the one who starts the multithreaded task, but I'm just calling another method that does. Since CallAnotherAsyncMethod() is also an async method, it must follow the exact same rules, right?. I can't have every async method just await another async sub-method forever, at some point it must stop unless you want infinite recursion.
The way I currently understand it, and I know this is wrong, is that async doesn't use multithreading, but it does require it, otherwise it's just a synchronous method lying to you.
So here's what might help. If async truly does not require multithreading, the following situation must be producible, I just can't find a way to do it. Can somebody create an example of a method that follows these rules:
Asynchronous.
Actually runs asynchronously.
Doesn't use any multithreading like calling Task.Run() or using a BackgroundWorker etc.
Doesn't call any other Async methods (unless you can also prove that this method follows these rules too).
Doesn't call any methods of the Task class like Task.Delay() (unless you can also prove that this method follows these rules too).
Any help or clarification would be really helpful! I feel like an idiot for not understanding this topic.

The easiest example of an async operation that does not use any kind of threads is waiting for a event to happen.
Create a UI app with your framework of choice and have two buttons, one called PrimeButton and one called RunButton
private TaskCompletionSource<object> _event = new TaskCompletionSource<object>();
//You are only allowed to do async void if you are writing a event handler!
public async void PrimeButton_OnClick(object sender, EventArgs e)
{
//I moved the code in to Example() so the async void would not be a distraction.
await Example();
}
public async Task Example()
{
await _event.Task;
MessageBox.Show("Run Clicked");
}
public async void RunButton_OnClick(object sender, EventArgs e)
{
_event.SetResult(null);
}
The await will wait till you click the 2nd button before it allows the code to continue and show the message box. No extra threads where involved at all here, all work was done using only the UI thread.
All a Task is, is a object that represents "something that will be finished at some point in the future". That something could be waiting for a background thread to complete that was started by a Task.Run or it could be waiting for a function to be called like the .SetResult( on the TaskCompletionSource<T>, or it could be waiting for some kind of disk or network IO to finish and be read in to a buffer by the OS (however internally this is usually implemented via a internal TaskCompletionSource<T> buried inside of the ReadAsync() function, so it is just a repeat of the last example with a wrapper around it)

Async controller method not returning

This code hangs (does not return a response) when I make a request to it:
public class MyController : ApiController
{
public async Task<IQueryable<int>> Get()
{
return await new Task<IQueryable<int>>(() => new List<int>().AsQueryable());
}
}
But this method works fine:
public IQueryable<int> Get()
{
return new List<int>().AsQueryable();
}
What fundamental knowledge am I missing??!

As the other answer noted, the reason your controller is not finishing is because the task is not started. However, using Task.Start, Task.Factory.StartNew, or Task.Run is not a good solution on ASP.NET.
The entire point of using async/await on ASP.NET is to free up a thread. However, if you await a task that is started via Start/StartNew/Run, then you're freeing up one thread by taking up another thread from the same thread pool. In this case, you not only lose all the benefits of async/await, but you actually decrease your scalability by regularly throwing off the ASP.NET thread pool heuristics.
There are two types of tasks, as I describe on my blog: Delegate Tasks (which represent some work executed on a thread) and Promise Tasks (which represent an event). You should avoid Delegate Tasks on ASP.NET, including any tasks that are "started" (Start/StartNew/Run).
Since you're returning an IQueryable<T>, I'm assuming that your actual underlying operation is a database query. If you're using EF6, then you have full support for asynchronous queries, which are properly implemented with Promise Tasks.

You're not actually starting your Task so it will wait for something that will never begin.
Instead use Task.Factory.StartNew which will create and start at the same time, or call Task#Start and await that call.
An overview of ways to start a task: http://dotnetcodr.com/2014/01/01/5-ways-to-start-a-task-in-net-c/

There is absolutely no need in async/await there, the method can look like:
public Task<IQueryable<int>> Get()
{
return Task.FromResult(new List<int>().AsQueryable());
}
If you really need it to be async, ok, you can always write something like:
public async Task<IQueryable<int>> Get()
{
return await Task.FromResult(new List<int>().AsQueryable());
}
which will introduce little overhead (a whole state machine will be generated by compiler).
Also, as others already stated, tasks returned from async methods should be hot (started)
Keep in mind, that Task.FromResult will return completed task and this case can be optimized by async/await generated code, writing Task.Run in this case is at least wierd
Read Task-based Asynchronous Pattern for more details

Write your own async method

I would like to know how to write your own async methods the "correct" way.
I have seen many many posts explaining the async/await pattern like this:
http://msdn.microsoft.com/en-us/library/hh191443.aspx
// Three things to note in the signature:
// - The method has an async modifier.
// - The return type is Task or Task<T>. (See "Return Types" section.)
// Here, it is Task<int> because the return statement returns an integer.
// - The method name ends in "Async."
async Task<int> AccessTheWebAsync()
{
// You need to add a reference to System.Net.Http to declare client.
HttpClient client = new HttpClient();
// GetStringAsync returns a Task<string>. That means that when you await the
// task you'll get a string (urlContents).
Task<string> getStringTask = client.GetStringAsync("http://msdn.microsoft.com");
// You can do work here that doesn't rely on the string from GetStringAsync.
DoIndependentWork();
// The await operator suspends AccessTheWebAsync.
// - AccessTheWebAsync can't continue until getStringTask is complete.
// - Meanwhile, control returns to the caller of AccessTheWebAsync.
// - Control resumes here when getStringTask is complete.
// - The await operator then retrieves the string result from getStringTask.
string urlContents = await getStringTask;
// The return statement specifies an integer result.
// Any methods that are awaiting AccessTheWebAsync retrieve the length value.
return urlContents.Length;
}
private void DoIndependentWork()
{
resultsTextBox.Text += "Working........\r\n";
}
This works great for any .NET Method that already implements this functionality like
System.IO opertions
DataBase opertions
Network related operations (downloading, uploading...)
But what if I want to write my own method that takes quite some time to complete where there just is no Method I can use and the heavy load is in the DoIndependentWork method of the above example?
In this method I could do:
String manipulations
Calculations
Handling my own objects
Aggregating, comparing, filtering, grouping, handling stuff
List operations, adding, removing, coping
Again I have stumbled across many many posts where people just do the following (again taking the above example):
async Task<int> AccessTheWebAsync()
{
HttpClient client = new HttpClient();
Task<string> getStringTask = client.GetStringAsync("http://msdn.microsoft.com");
await DoIndependentWork();
string urlContents = await getStringTask;
return urlContents.Length;
}
private Task DoIndependentWork()
{
return Task.Run(() => {
//String manipulations
//Calculations
//Handling my own objects
//Aggregating, comparing, filtering, grouping, handling stuff
//List operations, adding, removing, coping
});
}
You may notice that the changes are that DoIndependentWork now returns a Task and in the AccessTheWebAsync task the method got an await.
The heavy load operations are now capsulated inside a Task.Run(), is this all it takes?
If that's all it takes is the only thing I need to do to provide async Method for every single method in my library the following:
public class FooMagic
{
public void DoSomeMagic()
{
//Do some synchron magic...
}
public Task DoSomeMagicAsync()
{
//Do some async magic... ?!?
return Task.Run(() => { DoSomeMagic(); });
}
}
Would be nice if you could explain it to me since even a high voted question like this:
How to write simple async method? only explains it with already existing methods and just using asyn/await pattern like this comment of the mentioned question brings it to the point:
How to write simple async method?

Actual Answer
You do that using TaskCompletionSource, which has a Promise Task that doesn't execute any code and only:
"Represents the producer side of a Task unbound to a delegate, providing access to the consumer side through the Task property."
You return that task to the caller when you start the asynchronous operation and you set the result (or exception/cancellation) when you end it. Making sure the operation is really asynchronous is on you.
Here is a good example of this kind of root of all async method in Stephen Toub's AsyncManualResetEvent implementation:
class AsyncManualResetEvent
{
private volatile TaskCompletionSource<bool> _tcs = new TaskCompletionSource<bool>();
public Task WaitAsync() { return _tcs.Task; }
public void Set() { _tcs.TrySetResult(true); }
public void Reset()
{
while (true)
{
var tcs = _tcs;
if (!tcs.Task.IsCompleted ||
Interlocked.CompareExchange(ref _tcs, new TaskCompletionSource<bool>(), tcs) == tcs)
return;
}
}
}
Background
There are basically two reasons to use async-await:
Improved scalability: When you have I/O intensive work (or other inherently asynchronous operations), you can call it asynchronously and so you release the calling thread and it's capable of doing other work in the mean time.
Offloading: When you have CPU intensive work, you can call it asynchronously, which moves the work off of one thread to another (mostly used for GUI threads).
So most of the .Net framework's asynchronous calls support async out of the box and for offloading you use Task.Run (as in your example). The only case where you actually need to implement async yourself is when you create a new asynchronous call (I/O or async synchronization constructs for example).
These cases are extremely rare, which is why you mostly find answers that
"Only explains it with already existing methods and just using async/await pattern"
You can go deeper in The Nature of TaskCompletionSource

Would be nice if you could explain it to me: How to write simple async
method?
First, we need to understand what an async method means. When one exposes an async method to the end user consuming the async method, you're telling him: "Listen, this method will return to you quickly with a promise of completing sometime in the near future". That is what you're guaranteeing to your users.
Now, we need to understand how Task makes this "promise" possible. As you ask in your question, why simply adding a Task.Run inside my method makes it valid to be awaited using the await keyword?
A Task implements the GetAwaiter pattern, meaning it returns an object called an awaiter (Its actually called TaskAwaiter). The TaskAwaiter object implements either INotifyCompletion or ICriticalNotifyCompletion interfaces, exposing a OnCompleted method.
All these goodies are in turn used by the compiler once the await keyword is used. The compiler will make sure that at design time, your object implements GetAwaiter, and in turn use that to compile the code into a state machine, which will enable your program to yield control back to the caller once awaited, and resume when that work is completed.
Now, there are some guidelines to follow. A true async method doesn't use extra threads behind the scenes to do its job (Stephan Cleary explains this wonderfully in There Is No Thread), meaning that exposing a method which uses Task.Run inside is a bit misleading to the consumers of your api, because they will assume no extra threading involved in your task. What you should do is expose your API synchronously, and let the user offload it using Task.Run himself, controlling the flow of execution.
async methods are primarily used for I/O Bound operations, since these naturally don't need any threads to be consumed while the IO operation is executing, and that is why we see them alot in classes responsible for doing IO operations, such as hard drive calls, network calls, etc.
I suggest reading the Parallel PFX teams article Should I expose asynchronous wrappers for synchronous methods? which talks exactly about what you're trying to do and why it isn't recommended.

TL;DR:
Task.Run() is what you want, but be careful about hiding it in your library.
I could be wrong, but you might be looking for guidance on getting CPU-Bound code to run asynchronously [by Stephen Cleary]. I've had trouble finding this too, and I think the reason it's so difficult is that's kinda not what you're supposed to do for a library - kinda...
Article
The linked article is a good read (5-15 minutes, depending) that goes into a decent amount of detail about the hows and whys of using Task.Run() as part of an API vs using it to not block a UI thread - and distinguishes between two "types" of long-running process that people like to run asynchronously:
CPU-bound process (one that is crunching / actually working and needs a separate thread to do its work)
Truly asynchronous operation (sometimes called IO-bound - one that is doing a few things here and there with a bunch of waiting time in between actions and would be better off not hogging a thread while it's sitting there doing nothing).
The article touches on the use of API functions in various contexts, and explains whether the associated architectures "prefer" sync or async methods, and how an API with sync and async method signatures "looks" to a developer.
Answer
The last section "OK, enough about the wrong solutions? How do we fix this the right way???" goes into what I think you're asking about, ending with this:
Conclusion: do not use Task.Run in the implementation of the method; instead, use Task.Run to call the method.
Basically, Task.Run() 'hogs' a thread, and is thus the thing to use for CPU-bound work, but it comes down to where it's used. When you're trying to do something that requires a lot of work and you don't want to block the UI Thread, use Task.Run() to run the hard-work function directly (that is, in the event handler or your UI based code):
class MyService
{
public int CalculateMandelbrot()
{
// Tons of work to do in here!
for (int i = 0; i != 10000000; ++i)
;
return 42;
}
}
...
private async void MyButton_Click(object sender, EventArgs e)
{
await Task.Run(() => myService.CalculateMandelbrot());
}
But... don't hide your Task.Run() in an API function suffixed -Async if it is a CPU-bound function, as basically every -Async function is truly asynchronous, and not CPU-bound.
// Warning: bad code!
class MyService
{
public int CalculateMandelbrot()
{
// Tons of work to do in here!
for (int i = 0; i != 10000000; ++i)
;
return 42;
}
public Task<int> CalculateMandelbrotAsync()
{
return Task.Run(() => CalculateMandelbrot());
}
}
In other words, don't call a CPU-bound function -Async, because users will assume it is IO-bound - just call it asynchronously using Task.Run(), and let other users do the same when they feel it's appropriate. Alternately, name it something else that makes sense to you (maybe BeginAsyncIndependentWork() or StartIndependentWorkTask()).