We Keep Coding

Async method somehow yields control when it shouldn't

Consider the code below
static void Main(string[] args)
{
var ts = new CancellationTokenSource();
CancellationToken ct = ts.Token;
Task<string> task = Task.Run(() =>
{
ct.ThrowIfCancellationRequested();
var task2 = ActualAsyncTask();
while (!task2.IsCompleted)
{
var t = DateTime.Now;
while (DateTime.Now - t < TimeSpan.FromSeconds(1))
{
}
ct.ThrowIfCancellationRequested();
}
return task2;
}, ct);
Console.ReadLine();
ts.Cancel();
Console.ReadLine();
}
static async Task<string> ActualAsyncTask()
{
await Task.Delay(1000);
for(int i = 0; i < 100; ++i)
{
var t = DateTime.Now;
while (DateTime.Now - t < TimeSpan.FromSeconds(1))
{
}
Console.WriteLine("tick");
}
return "success";
}
It spawns a task that busy-waits for a cancellation request while an asynchronous method also busy-waits and prints some text into console.
When you let it run for a few seconds and then press enter, the task will throw the cancellation exception. While it's an interesting trick, I don't understand how the async method is able to yield control and make it possible.
Both the anonymous lambda within the task and the asynchronous method report to run on the same worker thread, which means they should be synchronous. In my understanding this setup should not throw the cancellation exception past the first 1 second await, because past that point there are no more awaits to allow the while loop of the anonymous lambda to gain control of the thread flow, yet somehow they seemingly both run in parallel using one thread.
If I remove the await command entirely, the execution becomes as expected - sending a cancellation request no longer interrupts the task. What am I missing?

Can I guarantee runnable task continuations have been run?

Here's a significantly reduced test case from a piece of code I'm working on:
var i = 0;
var taskCompletionSource = new TaskCompletionSource<object>();
var task = taskCompletionSource.Task;
Task.Run(async () =>
{
await task;
i = 1;
});
// Synchronously complete `task`
taskCompletionSource.SetResult(null);
// ???
Console.WriteLine(i);
Assuming that this code runs in the context of an async method, what should replace // ??? to ensure that this code prints 1 rather than 0?
I believe I understand why as written the program will always print 0 -- the code is executing synchronously and nothing has yielded to the scheduler. But I am surprised to learn that
await Task.Yield();
doesn't suffice. Somewhat ironically (but perfectly understandably, given that it involves no asynchronous execution) neither does
await task;
On the other hand,
await Task.Delay(1);
does seem to be enough, but I'm not clear on whether that's a guarantee or an accident of timing.
To ask the question one more way: is there any (reasonable) code I can write which will guarantee that all continuations for task have run before proceeding?

Can I guarantee runnable task continuations have been run?
By awaiting them.
var i = 0;
var taskCompletionSource = new TaskCompletionSource<object>();
var task = taskCompletionSource.Task;
var continuation = Task.Run(async () =>
{
await task;
i = 1;
});
// Synchronously complete `task`
taskCompletionSource.SetResult(null);
// wait for the continuation
await continuation;
// ouputs 1
Console.WriteLine(i);
That works if you're withing an asynchronous method. If you're not, make it asynchronous. You can technically block, too, (.Wait() instead of await) but that invites deadlocks, so be careful.

Assuming that this code runs in the context of an async method
var i = 0;
await Task.Run(async () =>
{
await task;
i = 1;
});
Console.WriteLine(i);
or
var i = 0;
await task;
i = 1;
Console.WriteLine(i);

tasks not run in enumerable.repeat

I don't understand why this test shows that the lambda has run a single time. I can only see how it might yield 0 or 10000. But only once? Ideally, I'd like ALL the tasks to be executed, just like the tooltip documentation on Task.WhenAll suggests.
[Fact]
public async Task FireLotsOfQueries()
{
var counter = 0;
var taskList = Enumerable.Repeat(Task.Run(async () =>
{
++counter;
await Task.Delay(1000);
}), 10000);
await Task.WhenAll(taskList);
Assert.Equal(10000, counter);
}
Result:
Xunit.Sdk.EqualException: Assert.Equal() Failure Expected: 10000
Actual: 1

The problem is you are not creating 1000 tasks. You are creating an enumerable that contains the same task 1000 times. Try this:
public async Task FireLotsOfQueries()
{
var counter = 0;
var taskList = Enumerable.Range(0, 10000)
.Select(_=> Task.Run(async () =>
{
++counter;
await Task.Delay(1000);
}));
await Task.WhenAll(taskList);
Assert.Equal(10000, counter);
}
You will definitely need some locking around counter as this version also fails, but counter will be a value closer to 10000.

Running Tasks in parallel

I am failing to understand why this doesn't seem to run the tasks in Parallel:
var tasks = new Task<MyReturnType>[mbis.Length];
for (int i = 0; i < tasks.Length; i++)
{
tasks[i] = CAS.Service.GetAllRouterInterfaces(mbis[i], 3);
}
Parallel.ForEach(tasks, task => task.Start());
By stepping through the execution, I see that as soon as this line is evaluated:
tasks[i] = CAS.Service.GetAllRouterInterfaces(mbis[i], 3);
The task starts. I want to add all the new tasks to the list, and then execute them in parallel.

If GetAllRouterInterfaces is an async method, the resulting Task will already be started (see this answer for further explanation).
This means that tasks will contain multiple tasks all of which are running in parallel without the subsequent call to Parallel.ForEach.
You may wish to wait for all the entries in tasks to complete, you can do this with an await Task.WhenAll(tasks);.
So you should end up with:
var tasks = new Task<MyReturnType>[mbis.Length];
for (int i = 0; i < tasks.Length; i++)
{
tasks[i] = CAS.Service.GetAllRouterInterfaces(mbis[i], 3);
}
await Task.WhenAll(tasks);
Update from comments
It seems that despite GetAllRouterInterfaces being async and returning a Task it is still making synchronous POST requests (presumably before any other await). This would explain why you are getting minimal concurrency as each call to GetAllRouterInterfaces is blocking while this request is made. The ideal solution would be to make an aynchronous POST request, e.g:
await webclient.PostAsync(request).ConfigureAwait(false);
This will ensure your for loop is not blocked and the requests are made concurrently.
Further update after conversation
It seems you are unable to make the POST requests asynchronous and GetAllRouterInterfaces does not actually do any asynchronous work, due to this I have advised the following:
Remove async from GetAllRouterInterfaces and change the return type to MyReturnType
Call GetAllRouterInterfaces in parallel like so
var routerInterfaces = mbis.AsParallel()
.Select(mbi => CAS.Service.GetAllRouterInterfaces(mbi, 3));

I don't know if I understand you the right way.
First of all, if GetAllRouterInterfaces is returns a Task you have to await the result.
With Parallel.ForEach you can't await tasks like as it is, but you can do something similar like this:
public async Task RunInParallel(IEnumerable<TWhatEver> mbisItems)
{
//mbisItems == your parameter that you want to pass to GetAllRouterInterfaces
//degree of cucurrency
var concurrentTasks = 3;
//Parallel.Foreach does internally something like this:
await Task.WhenAll(
from partition in Partitioner.Create(mbisItems).GetPartitions(concurrentTasks)
select Task.Run(async delegate
{
using (partition)
while (partition.MoveNext())
{
var currentMbis = partition.Current;
var yourResult = await GetAllRouterInterfaces(currentMbis,3);
}
}
));
}

creating a .net async wrapper to a sync request

I have the following situation (or a basic misunderstanding with the async await mechanism).
Assume you have a set of 1-20 web request call that takes a long time: findItemsByProduct().
you want to wrap it around in an async request, that would be able to abstract all these calls into one async call, but I can't seem to be able to do it without using more threads.
If I'm doing:
int total = result.paginationOutput.totalPages;
for (int i = 2; i < total + 1; i++)
{
await Task.Factory.StartNew(() =>
{
result = client.findItemsByProduct(i);
});
newList.AddRange(result.searchResult.item);
}
}
return newList;
problem here, that the calls don't run together, rather they are waiting one by one.
I would like all the calls to run together and than harvest the results.
as pseudo code, I would like the code to run like this:
forEach item {
result = item.makeWebRequest();
}
foreach item {
List.addRange(item.harvestResults);
}
I have no idea how to make the code to do that though..

Ideally, you should add a findItemsByProductAsync that returns a Task<Item[]>. That way, you don't have to create unnecessary tasks using StartNew or Task.Run.
Then your code can look like this:
int total = result.paginationOutput.totalPages;
// Start all downloads; each download is represented by a task.
Task<Item[]>[] tasks = Enumerable.Range(2, total - 1)
.Select(i => client.findItemsByProductAsync(i)).ToArray();
// Wait for all downloads to complete.
Item[][] results = await Task.WhenAll(tasks);
// Flatten the results into a single collection.
return results.SelectMany(x => x).ToArray();

Given your requirements which I see as:
Process n number of non-blocking tasks
Process results after all queries have returned
I would use the CountdownEvent for this e.g.
var results = new ConcurrentBag<ItemType>(result.pagination.totalPages);
using (var e = new CountdownEvent(result.pagination.totalPages))
{
for (int i = 2; i <= result.pagination.totalPages+1; i++)
{
Task.Factory.StartNew(() => return client.findItemsByProduct(i))
.ContinueWith(items => {
results.AddRange(items);
e.Signal(); // signal task is done
});
}
// Wait for all requests to complete
e.Wait();
}
// Process results
foreach (var item in results)
{
...
}

This particular problem is solved easily enough without even using await. Simply create each of the tasks, put all of the tasks into a list, and then use WhenAll on that list to get a task that represents the completion of all of those tasks:
public static Task<Item[]> Foo()
{
int total = result.paginationOutput.totalPages;
var tasks = new List<Task<Item>>();
for (int i = 2; i < total + 1; i++)
{
tasks.Add(Task.Factory.StartNew(() => client.findItemsByProduct(i)));
}
return Task.WhenAll(tasks);
}
Also note you have a major problem in how you use result in your code. You're having each of the different tasks all using the same variable, so there are race conditions as to whether or not it works properly. You could end up adding the same call twice and having one skipped entirely. Instead you should have the call to findItemsByProduct be the result of the task, and use that task's Result.

If you want to use async-await properly you have to declare your functions async, and the functions that call you also have to be async. This continues until you have once synchronous function that starts the async process.
Your function would look like this:
by the way you didn't describe what's in the list. I assume they are
object of type T. in that case result.SearchResult.Item returns
IEnumerable
private async Task<List<T>> FindItems(...)
{
int total = result.paginationOutput.totalPages;
var newList = new List<T>();
for (int i = 2; i < total + 1; i++)
{
IEnumerable<T> result = await Task.Factory.StartNew(() =>
{
return client.findItemsByProduct(i);
});
newList.AddRange(result.searchResult.item);
}
return newList;
}
If you do it this way, your function will be asynchronous, but the findItemsByProduct will be executed one after another. If you want to execute them simultaneously you should not await for the result, but start the next task before the previous one is finished. Once all tasks are started wait until all are finished. Like this:
private async Task<List<T>> FindItems(...)
{
int total = result.paginationOutput.totalPages;
var tasks= new List<Task<IEnumerable<T>>>();
// start all tasks. don't wait for the result yet
for (int i = 2; i < total + 1; i++)
{
Task<IEnumerable<T>> task = Task.Factory.StartNew(() =>
{
return client.findItemsByProduct(i);
});
tasks.Add(task);
}
// now that all tasks are started, wait until all are finished
await Task.WhenAll(tasks);
// the result of each task is now in task.Result
// the type of result is IEnumerable<T>
// put all into one big list using some linq:
return tasks.SelectMany ( task => task.Result.SearchResult.Item)
.ToList();
// if you're not familiar to linq yet, use a foreach:
var newList = new List<T>();
foreach (var task in tasks)
{
newList.AddRange(task.Result.searchResult.item);
}
return newList;
}