We Keep Coding

Running async functions in parallel from list of interfaces

I have a similair question to Running async methods in parallel in that I wish to run a number of functions from a list of functions in parallel.
I have noted in a number of comments online it is mentioned that if you have another await in your methods, Task.WhenAll() will not help as Async methods are not parallel.
I then went ahead and created a thread for each using function call with the below (the number of parallel functions will be small typically 1 to 5):
public interface IChannel
{
Task SendAsync(IMessage message);
}
public class SendingChannelCollection
{
protected List<IChannel> _channels = new List<IChannel>();
/* snip methods to add channels to list etc */
public async Task SendAsync(IMessage message)
{
var tasks = SendAll(message);
await Task.WhenAll(tasks.AsParallel().Select(async task => await task));
}
private IEnumerable<Task> SendAll(IMessage message)
{
foreach (var channel in _channels)
yield return channel.SendAsync(message, qos);
}
}
I would like to double check I am not doing anything horrendous with code smells or bugs as i get to grips with what I have patched together from what i have found online. Many thanks in advance.

Let's compare the behaviour of your line:
await Task.WhenAll(tasks.AsParallel().Select(async task => await task));
in contrast with:
await Task.WhenAll(tasks);
What are you delegating to PLINQ in the first case? Only the await operation, which does basically nothing - it invokes the async/await machinery to wait for one task. So you're setting up a PLINQ query that does all the heavy work of partitioning and merging the results of an operation that amounts to "do nothing until this task completes". I doubt that is what you want.
If you have another await in your methods, Task.WhenAll() will not help as Async methods are not parallel.
I couldn't find that in any of the answers to the linked questions, except for one comment under the question itself. I'd say that it's probably a misconception, stemming from the fact that async/await doesn't magically turn your code into concurrent code. But, assuming you're in an environment without a custom SynchronizationContext (so not an ASP or WPF app), continuations to async functions will be scheduled on the thread pool and possibly run in parallel. I'll delegate you to this answer to shed some light on that. That basically means that if your SendAsync looks something like this:
Task SendAsync(IMessage message)
{
// Synchronous initialization code.
await something;
// Continuation code.
}
Then:
The first part before await runs synchronously. If this part is heavyweight, you should introduce parallelism in SendAll so that the initialization code is run in parallel.
await works as usual, waiting for work to complete without using up any threads.
The continuation code will be scheduled on the thread pool, so if a few awaits finish up at the same time their continuations might be run in parallel if there's enough threads in the thread pool.
All of the above is assuming that await something actually awaits asynchronously. If there's a chance that await something completes synchronously, then the continuation code will also run synchronously.
Now there is a catch. In the question you linked one of the answers states:
Task.WhenAll() has a tendency to become unperformant with large scale/amount of tasks firing simultaneously - without moderation/throttling.
Now I don't know if that's true, since I weren't able to find any other source claiming that. I guess it's possible and in that case it might actually be beneficial to invoke PLINQ to deal with partitioning and throttling for you. However, you said you typically handle 1-5 functions, so you shouldn't worry about this.
So to summarize, parallelism is hard and the correct approach depends on how exactly your SendAsync method looks like. If it has heavyweight initialization code and that's what you want to parallelise, you should run all the calls to SendAsync in parallel. Otherwise, async/await will be implicitly using the thread pool anyway, so your call to PLINQ is redundant.

Should I use Task.Run to wait tasks in a synchronous context?

I have an ASPX page which I cannot convert to async but which uses some async methods in a synchronous context. The way it invokes them is like so:
public void MySyncMethod()
{
var myTask = Task.Run(() => _myField.DoSomethingAsync());
myTask.Wait();
//use myTask.Result
}
Is there any difference between doing that and the following as far as async/await and/or blocking goes?
public void MySyncMethod()
{
var myTask = _myField.DoSomethingAsync(); //just get the Task direct, no Task.Run
myTask.Wait();
//use myTask.Result
}
I assume a previous developer added the Task.Run for a reason. But I am having issues which accessing things in HttpContext as the work is being run on a different thread.
Is there a reason to use Task.Run here?

Is there any difference between doing that and the following as far as
async/await and/or blocking goes?
Yes the first block of code uses a thread pool thread then waits for this to return, so your using two threads not one. They both block.
I assume a previous developer added the Task.Run for a reason.
Yes, blocking (directly) on async code from an ASP.Net context is a bad idea and can cause deadlocks. So you second block of code is more efficent (in thread usage) but suffers from serious deadlock issues.
The correct solution here is to make public void MySyncMethod() async itself (public async Task MySyncMethod()). Both these solutions have drawbacks and the only real way out is to make the whole call stack async. If you can do this, do it.
If you can't call an async method from another async method then Task.Run is the way to go. See How to call asynchronous method from synchronous method in C#? for more details.
If you want HttpContext inside your thread have a read though Using HttpContext in Async Task I would definitely favour:
Make every thing async
Or the Read the values from the context then pass them
Options of those answers and keep in mind
First off, you're not creating a copy of the object, you're just
copying the reference to the object.HttpContext isn't a
struct.....etc

The internal workings of asynchronous code based on async/await is fundamentally different than tasks started by Task.Run. async/await tasks are promise based and depend on the caller cooperating with returning the execution back to the asynchronous method when appropriate. Tasks started by Task.Run however are usually started on a parallel thread taken from the thread pool and do not depend on the caller's cooperation to continue execution when appropriate.
This constellation leads to the problem that you can not treat a promise based task the same as the other tasks, since the promise based task might wait for the callers cooperation to return the execution, which might never occur since the other task is executed independently and might wait for the caller. The result is a deadlock.
The solution is a specific Task.Run overload that will create a proxy for an existing task-based method that allows proper execution of a promise based task. It is safe to call Wait on this proxy. That's why the other developer used this construct. He could have also simplified the call and avoided an anonymous method like this:
var myTask = Task.Run(_myField.DoSomethingAsync);

Task.Run is used to run code asynchronously.
Be clear that it returns Task and needs to be awaited. Here's an example:
Task myTask = Task.Run(() => DoSomething());
await myTask;

Why use async when I have to use await?

I've been stuck on this question for a while and haven't really found any useful clarification as to why this is.
If I have an async method like:
public async Task<bool> MyMethod()
{
// Some logic
return true;
}
public async void MyMethod2()
{
var status = MyMethod(); // Visual studio green lines this and recommends using await
}
If I use await here, what's the point of the asynchronous method? Doesn't it make the async useless that VS is telling me to call await? Does that not defeat the purpose of offloading a task to a thread without waiting for it to finish?

Does that not defeat the purpose of offloading a task to a thread without waiting for it to finish?
Yes, of course. But that's not the purpose of await/async. The purpose is to allow you to write synchronous code that uses asynchronous operations without wasting threads, or more generally, to give the caller a measure of control over the more or less asynchronous operations.
The basic idea is that as long as you use await and async properly, the whole operation will appear to be synchronous. This is usually a good thing, because most of the things you do are synchronous - say, you don't want to create a user before you request the user name. So you'd do something like this:
var name = await GetNameAsync();
var user = await RemoteService.CreateUserAsync(name);
The two operations are synchronous with respect to each other; the second doesn't (and cannot!) happen before the first. But they aren't (necessarily) synchronous with respect to their caller. A typical example is a Windows Forms application. Imagine you have a button, and the click handler contains the code above - all the code runs on the UI thread, but at the same time, while you're awaiting, the UI thread is free to do other tasks (similar to using Application.DoEvents until the operation completes).
Synchronous code is easier to write and understand, so this allows you to get most of the benefits of asynchronous operations without making your code harder to understand. And you don't lose the ability to do things asynchronously, since Task itself is just a promise, and you don't always have to await it right away. Imagine that GetNameAsync takes a lot of time, but at the same time, you have some CPU work to do before it's done:
var nameTask = GetNameAsync();
for (int i = 0; i < 100; i++) Thread.Sleep(100); // Important busy-work!
var name = await nameTask;
var user = await RemoteService.CreateUserAsync(name);
And now your code is still beautifuly synchronous - await is the synchronization point - while you can do other things in parallel with the asynchronous operations. Another typical example would be firing off multiple asynchronous requests in parallel but keeping the code synchronous with the completion of all of the requests:
var tasks = urls.Select(i => httpClient.GetAsync(i)).ToArray();
await Task.WhenAll(tasks);
The tasks are asynchronous in respect to each other, but not their caller, which is still beautifuly synchronous.
I've made a (incomplete) networking sample that uses await in just this way. The basic idea is that while most of the code is logically synchronous (there's a protocol to be followed - ask for login->verify login->read loop...; you can even see the part where multiple tasks are awaited in parallel), you only use a thread when you actually have CPU work to do. Await makes this almost trivial - doing the same thing with continuations or the old Begin/End async model would be much more painful, especially with respect to error handling. Await makes it look very clean.

If I use await here, what's the point of the asynchronous method?
await does not block thread. MyMethod2 will run synchronously until it reaches await expression. Then MyMethod2 will be suspended until awaited task (MyMethod) is complete. While MyMethod is not completed control will return to caller of MyMethod2. That's the point of await - caller will continue doing it's job.
Doesn't it make the async useless that VS is telling me to call await?
async is just a flag which means 'somewhere in the method you have one or more await'.
Does that not defeat the purpose of offloading a task to a thread
without waiting for it to finish?
As described above, you don't have to wait for task to finish. Nothing is blocked here.
NOTE: To follow framework naming standards I suggest you to add Async suffix to asynchronous method names.

An async method is not automatically executed on a different thread. Actually, the opposite is true: an async method is always executed in the calling thread. async means that this is a method that can yield to an asynchronous operation. That means it can return control to the caller while waiting for the other execution to complete. So asnync methods are a way to wait for other asynchronoous operations.
Since you are doing nothing to wait for in MyMethod2, async makes no sense here, so your compiler warns you.
Interestingly, the team that implemented async methods has acknowledged that marking a method async is not really necessary, since it would be enough to just use await in the method body for the compiler to recognize it as async. The requirement of using the async keyword has been added to avoid breaking changes to existing code that uses await as a variable name.

C# async tasks: do all participating methods have to be async as well?

When we are dealing with async tasks, the task consists of multiple method calls (possibly of other classes) which in turn consist of yet further method calls. We could call this a tree. Do all methods have to be of type async task for the original task to truly behave async?
Would the answer be true for any kind of background task of other similar languages, mostly Java as well?
(In C#, once I had one of my methods somewhere in this tree get into an infinite loop (talk about a long running task!), yet sadly, the original async task's isRunning property returned false.)

Do all methods have to be of type async task for the original task to truly behave async?
In order to behave asynchronously, an async method must have an await statement for an operation that is not already completed.
I think my async intro blog post would be helpful for you.
Would the answer be true for any kind of background task of other similar languages, mostly Java as well?
Yes.
sadly, the original async task's isRunning property returned false
I assume you mean that the TaskStatus was not TaskStatus.Running. This is normal and expected, since asynchronous tasks are Promise Tasks, not Delegate Tasks (see my blog for more info).

The short answer is no. Async means that work being done by another process can be handed off to that process and so not block the calling thread. All the better if those processes don’t reside on the same machine, since that work will occupy very little compute or IO at the caller. This is achieved by suspending method execution using the async/await mechanism, which promises to resume on some synchronisation context once completed. So, why the overview?
Because this means that we can and should judiciously choose those operations that should be async, and so apply the await keyword in an async method. Some won’t need it, since they are lightweight and adding async to them may even add compute time to them, due to the creation of the async state machine. So the operations in a method can often be a mixture of asynchronous and synchronous code.
And this in turn means that some methods would use the Task return type with async/await, and some will be regular, synchronous methods.

No, they don't have to. Simple example, in a Forms application this
button.Click += async (_, __) =>
{
await Task.Run(() => MyClass.SomeTimeconsumingMethod());
};
still lets you move the window during the execution of SomeTimeconsumingMethod, whereas this one
button.Click += (_, __) =>
{
MyClass.SomeTimeconsumingMethod();
};
doesn't let you do it.

The only requirement for using the await keyword is that the method you are awaiting returns a Task or Task<T>.
The exact implementation of this method (async or return someTask) has no influence on the correctness of the code, although it may influence the function.
Any normal code can run in an async method alongside your await calls. The only significant restriction is that out and ref are not allowed.

Do all methods have to be of type async task for the original task to truly behave async
It depends on what do you mean by "all" word.
Some APIs must not be asynchronous. Think about List<T>.Add, for example. Do you really want it to be asynchronous?
Some APIs should be asynchronous, but there's no asynchronous counterpart (e.g. SMO, or File static class).
Ideally you should do "async all the way", when talking about IO-bound operations. But IRL you can face with violation of this rule, when some part of your async method chain will execute IO-bound operation synchronously.

Are continuations queued on a Task always executed on the thread that finishes the task?

I'm trying to implement coroutines using async/await, and for that I want to ensure my coroutines are only executing on one thread (the thread that resumes them).
I am currently using a custom awaiter which simply queues the continuation on the coroutine object. When a coroutine wants to yield, it awaits this custom awaiter. When a coroutine is resumed, it simply calls the continuation.
I can guarantee that only one continatuion is queued per resume, ie. that we don't create multiple tasks without awaiting them. I can also guarantee that we will only be awaiting tasks that ultimately await the custom awaiter or other tasks that await the custom awaiter. That is, we won't be awating any "external" tasks.
An example would be something like this:
private static async Task Sleep(int ms)
{
Stopwatch timer = Stopwatch.StartNew();
do
{
await Coroutine.Yield();
}
while (timer.ElapsedMilliseconds < ms);
}
private static async Task Test()
{
// Second resume
await Sleep(1000);
// Unknown how many resumes
}
private static async Task Main()
{
// First resume
await Coroutine.Yield();
// Second resume
await Test();
}
It all seems to work and it seems like the continuations to the tasks are indeed executed inline on the same thread. I just want to be sure that this behavior is consistent and that I can rely on it.
I was able to check the reference source, and have found what I think is the place continuations are executed. The path to this function seems pretty complex though, and I cannot determine just what call exactly leads to this function being called (but I assume it is some compiler generated call).
Now, from this function, it seems like the continuation is not inlined if:
The current thread is aborting
This should not be a problem, as the current thread is willingly executing the coroutine, and we shouldn't be executing a coroutine if we're aborting.
IsValidLocationForInlining is false
This property is false if the current synchronization context is non default, or the current task scheduler is non default. As a precaution I am doing SynchronizationContext.SetSynchronizationContext(null) for the duration of the continuation, when resuming a coroutine. I will also be ensuring that the task scheduler is the default.
Now, my actual question is if I can rely on this behavior. Is this something that is likely to change in .NET versions? Would it be better to implement a custom synchronization context which ensured that all continuations were run by the coroutine?
Furthermore, I know the task libraries have changed a lot from .NET 4 to .NET 4.5. The reference source is for .NET 4.5, as far as I know, so I want to know if someone knows if this behavior has changed. I will be using the coroutines library on .NET 4.0 with Microsoft.Bcl.Async mainly, and it also seems to work fine here.

I'm trying to implement coroutines using async/await, and for that I
want to ensure my coroutines are only executing on one thread (the
thread that resumes them).
I think you can safely rely upon this behavior. This should be true as long as you do not use any of the following features:
Custom TPL task schedulers;
Custom synchronization contexts;
ConfiguredTaskAwaitable or Task.ConfigureAwait();
YieldAwaitable or Task.Yield();
Task.ContinueWith();
Anything which may lead to a thread switch, like an async I/O API, Task.Delay(), Task.Run(), Task.Factory.StartNew(), ThreadPool.QueueUserWorkItem() etc.
The only thing that you use here is TaskAwaiter, more about it below.
First of all, you should not be worried about a thread switch inside the task, where you do await Coroutine.Yield(). The code will be resumed exactly on the same thread where you explicitly call the continuation callback, you have complete control over this.
Secondly, the only Task object you have here is that generated by the state machine logic (specifically, by AsyncTaskMethodBuilder). This is the task returned by Test(). As mentioned above, a thread switch inside this task may not take place, unless you do it explicitly before calling the continuation callback via your custom awaiter.
So, your only remaining concerned is about a thread switch which may happen at the point where you're awaiting the result of the task returned by Test(). That's where TaskAwaiter comes into play.
The behavior of TaskAwaiter is undocumented. As far as I can tell from the Reference Sources, IsValidLocationForInlining is not observed for the TaskContinuation kind of continuation (created by TaskAwaiter). The present behavior is the following: the continuation will be not inlined if the current thread was aborted or if the current thread's synchronization context is different from that captured by TaskAwaiter.
If you don't want to rely upon this, you can create another custom awaiter to replace TaskAwaiter for your coroutine tasks. You could implement it using Task.ContinueWith( TaskContinuationOptions.ExecuteSynchronously), which behavior is unofficially documented by Stephen Toub in his "When "“ExecuteSynchronously” doesn't execute synchronously" blog post. To sum up, the ExecuteSynchronously continuation won't be inlined under the following severe conditions:
the current thread was aborted;
the current thread has stack overflows;
the target task scheduler rejects inlining (but you're not using custom task schedulers; the default one always promotes inlining where possible).