We Keep Coding

Async method somehow yields control when it shouldn't

Consider the code below
static void Main(string[] args)
{
var ts = new CancellationTokenSource();
CancellationToken ct = ts.Token;
Task<string> task = Task.Run(() =>
{
ct.ThrowIfCancellationRequested();
var task2 = ActualAsyncTask();
while (!task2.IsCompleted)
{
var t = DateTime.Now;
while (DateTime.Now - t < TimeSpan.FromSeconds(1))
{
}
ct.ThrowIfCancellationRequested();
}
return task2;
}, ct);
Console.ReadLine();
ts.Cancel();
Console.ReadLine();
}
static async Task<string> ActualAsyncTask()
{
await Task.Delay(1000);
for(int i = 0; i < 100; ++i)
{
var t = DateTime.Now;
while (DateTime.Now - t < TimeSpan.FromSeconds(1))
{
}
Console.WriteLine("tick");
}
return "success";
}
It spawns a task that busy-waits for a cancellation request while an asynchronous method also busy-waits and prints some text into console.
When you let it run for a few seconds and then press enter, the task will throw the cancellation exception. While it's an interesting trick, I don't understand how the async method is able to yield control and make it possible.
Both the anonymous lambda within the task and the asynchronous method report to run on the same worker thread, which means they should be synchronous. In my understanding this setup should not throw the cancellation exception past the first 1 second await, because past that point there are no more awaits to allow the while loop of the anonymous lambda to gain control of the thread flow, yet somehow they seemingly both run in parallel using one thread.
If I remove the await command entirely, the execution becomes as expected - sending a cancellation request no longer interrupts the task. What am I missing?

Task<Task> illegal in C#?

This code gives a compile time error in VS2015
Error CS0266 Cannot implicitly convert type 'System.Threading.Tasks.Task' to 'System.Threading.Tasks.Task<System.Threading.Tasks.Task>'. An explicit conversion exists (are you missing a cast?)
UPDATE 1: extended code after a Task instead of Task< Task > was suggested.
int i = 0;
Task<int> test = Task.Run( () => {
return i;
} );
i = test.Result;
Task t = Task.Run( () => { } );
Task<Task> test2 = Task.Run( () => {
return t;
} );
t = test2.Result;
What am I doing wrong?
UPDATE 2:
This code gives a warning (and I want no warnings and no suppress pragmas)
Warning CS1998 This async method lacks 'await' operators and will run synchronously. Consider using the 'await' operator to await non-blocking API calls, or 'await Task.Run(...)' to do CPU-bound work on a background thread.
int i = 0;
Task<int> test = Task.Run( () => {
return i;
} );
i = test.Result;
Task t = Task.Run( () => { } );
Task<Task> test2 = Task.Run( async () => {
return t;
} );
t = test2.Result;
UPDATE 3:
To all people insisting on Task test2.
StartAndReturnSecondTaskAsync must return the second task (executing longOperation2)
StartAndReturnSecondTaskAsync must be async, i.e. UI must not block for the duration of longOperation1
public static async Task<Task> StartAndReturnSecondTaskAsync() {
Task t = await Task.Run( () => {
return StartAndReturnSecondTask();
} );
return t;
}
public static Task StartAndReturnSecondTask() {
var importantData = longOperation1();
return Task.Run( () => {
longOperation2( importantData );
} );
}
...
Task second = await StartAndReturnSecondTaskAsync();

It's an extremely common but to use Task.Run to perform some asychronous operation, and if using the older method StartNew it would do what you're expecting, which is to schedule a thread pool thread to start the asynchronous operation, and tell you when that asynchronous operation has finished being started.
This is, however, basically never what people actually want. They want to know when the asynchronous operation called in Task.RUn finishes. Because of this, Task.Run will unwrap any Task<Task> that would be returned, so that what you're really seeing is the innerTask`.
If you really do just want to have a thread pool thread start the task, and just know when it has finished being started, then you can use Task.Factory.StartNew which does not unwrap the Task<Task> on your behalf.

Task<Task> are quite common and often a problem (if we erroneously wait/await wrong, outer task):
int i = 0;
Task<int> test = Task.Run(() => {
return i;
});
Task t = Task.Run(() => {
});
// please, notice "async"
Task<Task> test2 = Task.Run(async () => { // <- async: let insist on Task<Task>...
return t;
});
There's even an extension method for Task<Task>
Task backToTask = test2.Unwrap();
In your case you want
Task test2 = Task.Run(() => {
return t;
});
Since Task.Run calls Unwrap() for you

A simple solution without getting into much details:
The code that would work would be:
int i = 0;
Task<int> test = Task.Run(() =>
{
return i;
});
Task t = Task.Run(() => { });
Task test2 = Task.Run(() =>
{
return t;
});
If you go with other possible approaches that use await, You would need to make the context async, like:
public async Task DoesTheThreadedOperations()
{
// Your code in question and other code
}
As per your Update 3
public static async Task StartAndReturnSecondTaskAsync()
{
await Task.Run(() =>
{
return StartAndReturnSecondTask();
});
}
public static Task StartAndReturnSecondTask()
{
System.Threading.Thread.Sleep(5000);
return Task.Run(() =>
{
System.Threading.Thread.Sleep(10000);
});
}
I have used sleep to imitate your long operations. I hope you don't necessarily want to use Task<Task>. This should give you enough thrust to attack your scenario. You can further improve it to get your exact results by making these two operations two different threads and pass the results from one to other, await the one you want and run and leave the other longer task running without await. That is, call both methods from your main method, one awaited and other not.

Running Tasks in parallel

I am failing to understand why this doesn't seem to run the tasks in Parallel:
var tasks = new Task<MyReturnType>[mbis.Length];
for (int i = 0; i < tasks.Length; i++)
{
tasks[i] = CAS.Service.GetAllRouterInterfaces(mbis[i], 3);
}
Parallel.ForEach(tasks, task => task.Start());
By stepping through the execution, I see that as soon as this line is evaluated:
tasks[i] = CAS.Service.GetAllRouterInterfaces(mbis[i], 3);
The task starts. I want to add all the new tasks to the list, and then execute them in parallel.

If GetAllRouterInterfaces is an async method, the resulting Task will already be started (see this answer for further explanation).
This means that tasks will contain multiple tasks all of which are running in parallel without the subsequent call to Parallel.ForEach.
You may wish to wait for all the entries in tasks to complete, you can do this with an await Task.WhenAll(tasks);.
So you should end up with:
var tasks = new Task<MyReturnType>[mbis.Length];
for (int i = 0; i < tasks.Length; i++)
{
tasks[i] = CAS.Service.GetAllRouterInterfaces(mbis[i], 3);
}
await Task.WhenAll(tasks);
Update from comments
It seems that despite GetAllRouterInterfaces being async and returning a Task it is still making synchronous POST requests (presumably before any other await). This would explain why you are getting minimal concurrency as each call to GetAllRouterInterfaces is blocking while this request is made. The ideal solution would be to make an aynchronous POST request, e.g:
await webclient.PostAsync(request).ConfigureAwait(false);
This will ensure your for loop is not blocked and the requests are made concurrently.
Further update after conversation
It seems you are unable to make the POST requests asynchronous and GetAllRouterInterfaces does not actually do any asynchronous work, due to this I have advised the following:
Remove async from GetAllRouterInterfaces and change the return type to MyReturnType
Call GetAllRouterInterfaces in parallel like so
var routerInterfaces = mbis.AsParallel()
.Select(mbi => CAS.Service.GetAllRouterInterfaces(mbi, 3));

I don't know if I understand you the right way.
First of all, if GetAllRouterInterfaces is returns a Task you have to await the result.
With Parallel.ForEach you can't await tasks like as it is, but you can do something similar like this:
public async Task RunInParallel(IEnumerable<TWhatEver> mbisItems)
{
//mbisItems == your parameter that you want to pass to GetAllRouterInterfaces
//degree of cucurrency
var concurrentTasks = 3;
//Parallel.Foreach does internally something like this:
await Task.WhenAll(
from partition in Partitioner.Create(mbisItems).GetPartitions(concurrentTasks)
select Task.Run(async delegate
{
using (partition)
while (partition.MoveNext())
{
var currentMbis = partition.Current;
var yourResult = await GetAllRouterInterfaces(currentMbis,3);
}
}
));
}

Async/Await action within Task.Run()

Task.Run(()=>{}) puts the action delegate into the queue and returns the task .
Is there any benefit of having async/await within the Task.Run()?
I understand that Task.Run() is required since if we want to use await directly, then the calling method will need to be made Async and will affect the calling places.
Here is the sample code which has async await within Task.Run(). The full sample is provided here: Create pre-computed tasks.
Task.Run(async () => { await new WebClient().DownloadStringTaskAsync("");});
Alternatively this could have been done:
Task.Run(() => new WebClient().DownloadStringTaskAsync("").Result;);
Since both, Task.Run() and Await will queue the work and will be picked by the thread pool, could the async/await within the Task.Run() be a bit redundant?

Is there any benefit of having async/await within the Task.Run() ?
Yes. Task.Run runs some action on a thread-pool thread. If such action does some IO work and asynchronously waits for the IO operation to complete via await, then this thread-pool thread can be used by the system for other work while the IO operation is still running.
Example:
Task.Run( async () =>
{
DoSomeCPUIntensiveWork();
// While asynchronously waiting for this to complete,
// the thread is given back to the thread-pool
var io_result = await DoSomeIOOperation();
DoSomeOtherCPUIntensiveWork(io_result);
});

Is there any benefit of having async/await within the Task.Run()
An async method returns to the caller as soon as the first await is hit (that operates on a non-completed task). So if that first execution "streak" of an async method takes a long time Task.Run will alter behavior: It will cause the method to immediately return and execute that first "streak" on the thread-pool.
This is useful in UI scenarios because that way you can make 100% sure that you are not blocking the UI. Example: HttpWebRequestdoes DNS resolution synchronously even when you use one of the async methods (this is basically a library bug/design error). This can pause the UI thread. So you can use Task.Run to be 100% sure that the UI is never blocked for longer than a few microseconds.
So back to the original question: Why await inside a Task.Run body? For the same reason you normally await: To unblock the thread.

In the example that you linked the main thread is being blocked until the asynchronous operation is done. It's being blocked by calling Wait() (which by the way is generally a bad idea).
Let's have a look at the return from the DownloadStringAsync in the linked sample:
return Task.Run(async () =>
{
content = await new WebClient().DownloadStringTaskAsync(address);
cachedDownloads.TryAdd(address, content);
return content;
});
Why would you wrap this in a Task? Think about your options for a second. If you don't want to wrap this in a Task, how would you make sure the method returns a Task<string> and still have it work? You'd mark the method as async of course! However, if you mark your method as async and you call Wait on it, you'll most likely end up with a deadlock, since the main thread is waiting for the work to finish, and your blocking the main thread so it can't let you know it's done.
When marking a method as async, the state machine will run on the calling thread, in your example however, the state machine runs on a separate thread, meaning there is little to no work being done on the main thread.

Calling Async from Non-Async Methods
We do some stuff like that when we are trying to call an async method inside of a non-async method. Especially if the async method is a known quantity. We use more of a TaskFactory though ... fits a pattern, makes it easier to debug, makes sure everyone takes the same approach (and -- gives us one throat to choke if async-->sync starts acting buggy).
So, Your Example
Imagine, in your example, that you have a non-async function. And, within that function, you need to call await webClient.DoSomethingAsync(). You can't call await inside of a function that's not async -- the compiler won't let you.
Option 1: Zombie Infestation
Your first option is to crawl all the way back up your call stack, marking every da*n method along the way as async and adding awaits everywhere. Everywhere. Like, everywhere. Then -- since all those methods are now async, you need to make the methods that reference them all async.
This, btw, is probably the approach many of the SO enthusiasts are going to advocate. Because, "blocking a thread is a bad idea."
So. Yeah. This "let async be async" approach means your little library routine to get a json object just reached out and touched 80% of the code. Who's going to call the CTO and let him know?
Option 2: Just Go with the One Zombie
OR, you can encapsulate your async inside of some function like yours...
return Task.Run(async () => {
content = await new WebClient().DoSomethingAsync();
cachedDownloads.TryAdd(address, content);
return content;
});
Presto... the zombie infestation has been contained to a single section of code. I'll leave it to the bit-mechanics to argue over how/why that gets executed at the CPU-level. I don't really care. I care that nobody has to explain to the CTO why the entire library should now be 100% async (or something like that).

Confirmed, wrapping await with Task.Run use 2 threads instead of one.
Task.Run(async () => { //thread #1
await new WebClient().DownloadStringTaskAsync(""); //thread #2
});
Say you have four calls wrapped like this, it will use 4 x 2 = 8 threads.
It would be better to just call these with simple await instead. For example:
Task<byte[]> t1 = new WebClient().DownloadStringTaskAsync("");
Task<byte[]> t2 = new WebClient().DownloadStringTaskAsync("");
byte[] t1Result = await t1;
byte[] t2Result = await t2;
Here is the proof that wrapped Task.Run are using extra threads. (Not using WebClient to prove the point)
private static async Task wrapped()
{
List<Task> tasks = new List<Task>();
tasks.AddRange(new []
{
Task.Run(async() => await new MyThread().RunMe()),
Task.Run(async() => await new MyThread().RunMe()),
Task.Run(async() => await new MyThread().RunMe()),
Task.Run(async() => await new MyThread().RunMe()),
});
Thread.Sleep(1000);
int number = Process.GetCurrentProcess().Threads.Count;
Console.WriteLine($"While running thread count: {number}");
await Task.WhenAll(tasks);
}
Unwrapped
private static async Task unwrapped()
{
List<Task> tasks = new List<Task>();
Task<int> t1 = new MyThread().RunMe();
Task<int> t2 = new MyThread().RunMe();
Task<int> t3 = new MyThread().RunMe();
Task<int> t4 = new MyThread().RunMe();
tasks.AddRange(new[] {t1, t2, t3, t4});
Thread.Sleep(1000);
int number = Process.GetCurrentProcess().Threads.Count;
Console.WriteLine($"While running thread count: {number}");
int i1 = await t1;
int i2 = await t2;
int i3 = await t3;
int i4 = await t4;
}
Full POC code here
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
namespace AsyncThreadDemo
{
class Program
{
static async Task Main(string[] args)
{
int number = Process.GetCurrentProcess().Threads.Count;
Console.WriteLine($"Init thread count: {number}");
//await wrapped();
await unwrapped();
number = Process.GetCurrentProcess().Threads.Count;
Console.WriteLine($"Done thread count: {number}");
Console.ReadLine();
}
private static async Task wrapped()
{
List<Task> tasks = new List<Task>();
tasks.AddRange(new []
{
Task.Run(async() => await new MyThread().RunMe()),
Task.Run(async() => await new MyThread().RunMe()),
Task.Run(async() => await new MyThread().RunMe()),
Task.Run(async() => await new MyThread().RunMe()),
});
Thread.Sleep(1000);
int number = Process.GetCurrentProcess().Threads.Count;
Console.WriteLine($"While running thread count: {number}");
await Task.WhenAll(tasks);
}
private static async Task unwrapped()
{
List<Task> tasks = new List<Task>();
Task<int> t1 = new MyThread().RunMe();
Task<int> t2 = new MyThread().RunMe();
Task<int> t3 = new MyThread().RunMe();
Task<int> t4 = new MyThread().RunMe();
tasks.AddRange(new[] {t1, t2, t3, t4});
Thread.Sleep(1000);
int number = Process.GetCurrentProcess().Threads.Count;
Console.WriteLine($"While running thread count: {number}");
int i1 = await t1;
int i2 = await t2;
int i3 = await t3;
int i4 = await t4;
}
}
public class MyThread
{
public static int _counter;
public async Task<int> RunMe()
{
await Task.Run(() =>
{
for (int i = 0; i < 2; ++i)
{
Thread.Sleep(1000);
Console.WriteLine($"T{Thread.CurrentThread.ManagedThreadId} {i}");
}
Console.WriteLine($"T{Thread.CurrentThread.ManagedThreadId} done");
});
return _counter++;
}
}
}

Why Task.Delay doesn`t work in this situation

I'm testing the async and I found this situation that I can't understand:
var watch = Stopwatch.StartNew();
var t1 = Task.Factory.StartNew(async () =>
{
await Task.Delay(2000);
return 2;
});
var t2 = Task.Factory.StartNew(() =>
{
Task.Delay(1000);
return 1;
});
await Task.WhenAll(t1, t2);
var result = watch.ElapsedMilliseconds;
I would like to understand why the result is always 0! Why is not 1000, 2000 or the sum of the two tasks 3000? Why doesn't Task.WhenAll wait for the completion of the tasks?

Okay, so, the second one is the easy one, so let's handle that one.
For the second task, t2, you don't do anything with the result of Task.Delay(1000). You don't await it, you don't Wait it, etc. Given that the method is not async I supposed you meant for it to be a blocking wait. To do that you'd want to add Wait() to the end of the Delay call to make it a blocking wait, or just use Thread.Sleep().
For the first task, you're being bitten by the fact that you're using var. It's clearer what's happening when you don't use var:
Task<Task<int>> t1 = Task.Factory.StartNew(async () =>
{
await Task.Delay(2000);
return 2;
});
You're returning a task of task of int, not just a Task of int. The outer task is "completed" as soon as the inner task finishes being started. When you use WhenAll you don't care about when the outer task finishes, you care about when the inner task finishes. There are a number of ways to handle this. One is to use Unwrap.
Task<int> t1 = Task.Factory.StartNew(async () =>
{
await Task.Delay(2000);
return 2;
}).Unwrap();
Now you have your expected Task<int> and WhenAll will take at least 2000 milliseconds, as expected. You could also add in another await call to do the same thing:
Task<int> t1 = await Task.Factory.StartNew(async () =>
{
await Task.Delay(2000);
return 2;
});
As mentioned by svick in a comment another option would be to just use Task.Run instead of StartNew. Task.Run has a special set of overloads for methods that take a Func<Task<T>> and return a Task<T> and automatically unwrap them for you:
Task<int> t1 = Task.Run(async () =>
{
await Task.Delay(2000);
return 2;
});
For this reason it's preferable to use Task.Run as the default option when you're creating async lambdas as it will "handle" this issue for you, although it's best to be aware of it for the complex cases where you can't use Task.Run.
Finally we come to the option that you didn't do, which is what you probably should be actually doing in this case. Since Task.Delay already returns a Task, there's no need to put it in StartNew in the first place. Rather than creating a nested task and using Unwrap you can just not wrap it in the first place:
var t3 = Task.Delay(3000);
await Task.WhenAll(t1, t2, t3);
If you actually just want to wait for a fixed amount of time that's what you should be doing.