We Keep Coding

Correctly awaiting Task.Run with async lambda expression

so I have an async method wrapped in Task.Run like so:
//SynchronisationContext = UI Thread
await Task.Run(async =>
{
for (int i = 0; i < 100; i++)
{
var res = ComplicatedCalulation(); //Takes about 1 second
await ThirdPartyLib.WriteToDatabase(res);
}
});
Does the await on the above code await the async lambda code or does it just await the Task being started (i.e. returning straight away)
I know with Task.Factory.StartNew the correct use is
await Task.Factory.StartNew<Task>(async => await MyAsyncMethod()).Result
Is this also true for Task.Run?

Your code will wait till the end of the compute using the task mechanism. This will not block your thread but it will not execute code beyond this point until everything in your loop is done.
public static async Task Test()
{
await Task.Run(async () =>
{
for (int i = 0; i < 10; i++)
{
await Task.Delay(TimeSpan.FromSeconds(1));
Console.WriteLine($"inside {i}");
}
});
Console.WriteLine("Done");
}
will give the following output:
inside 0
inside 1
inside 2
inside 3
inside 4
inside 5
inside 6
inside 7
inside 8
inside 9
Done

Parallel start of several tasks, each of which has its own timeout

I have several similar tasks, each of which is limited to a timeout, i.e. must be completed faster than the specified time or return an empty result. The main purpose of these tasks is receiving a response from the server with a timeout limit.
An example of such task is below:
public async Task<List<Data>> GetDataWithTimeoutAsync(InputData data, int timeout)
{
List<Data> resultData = new List<Data>;
await Task.WhenAny(Task.Run(async () =>
{
resultData.Add(SomeWork(data));
}),
Task.Delay(timeout));
return resultData;
}
Each of these tasks works correctly separately.
But I want to run some of such tasks in parallel. For this, I use the following code.
public async Task<List<List<Data>>> GetAllDataAsync()
{
var resultTasks = new ConcurrentBag<Task<List<Data>>>();
var firtsTask = GetDataWithTimeoutAsync(firstInputData, firtsTimeout);
var secondTask = GetDataWithTimeoutAsync(secondInputData, secondTimeout);
var thirdTask = GetDataWithTimeoutAsync(thirdInputData, thirdTimeout);
resultTasks.Add(Task.Run(() => firtsTask));
resultTasks.Add(Task.Run(() => secondTask));
resultTasks.Add(Task.Run(() => thirdTask));
await Task.WhenAll(resultTasks);
var result = resultTasks.Select(t => t.Result).ToList();
return result;
}
But this code works incorrectly if different timeouts are set for nested tasks. In this case all of tasks are completed after the smallest of the timeouts.
How i can run some tasks in parallel with WhenAll if each of task is a result of WhenAny?

Your code doesn't compile, so I wrote something similar. I can't reproduce your results. In my case the WhenAll with different timeouts works as expected. It completes when the longest running task is completed, which is the second one (200 msec).
public static async Task Main(string[] args)
{
var task1 = GetDataAsync(100).WithTimeout(50); // Should timeout after 50 msec
var task2 = GetDataAsync(200).WithTimeout(300); // Should complete after 200 msec
var task3 = GetDataAsync(300).WithTimeout(100); // Should timeout after 100 msec
var stopwatch = Stopwatch.StartNew();
var results = await Task.WhenAll(task1, task2, task3); // Wait for all
stopwatch.Stop();
Console.WriteLine($"Results: {String.Join(", ", results)}");
Console.WriteLine($"Elapsed: {stopwatch.ElapsedMilliseconds} msec");
}
private static async Task<int> GetDataAsync(int input) // the input is used as delay
{
await Task.Delay(input);
return input;
}
public static Task<T> WithTimeout<T>(this Task<T> task, int timeout)
{
var delayTask = Task.Delay(timeout).ContinueWith(_ => default(T),
TaskContinuationOptions.ExecuteSynchronously);
return Task.WhenAny(task, delayTask).Unwrap();
}
Output:
Results: 0, 200, 0
Elapsed: 211 msec

Can I guarantee runnable task continuations have been run?

Here's a significantly reduced test case from a piece of code I'm working on:
var i = 0;
var taskCompletionSource = new TaskCompletionSource<object>();
var task = taskCompletionSource.Task;
Task.Run(async () =>
{
await task;
i = 1;
});
// Synchronously complete `task`
taskCompletionSource.SetResult(null);
// ???
Console.WriteLine(i);
Assuming that this code runs in the context of an async method, what should replace // ??? to ensure that this code prints 1 rather than 0?
I believe I understand why as written the program will always print 0 -- the code is executing synchronously and nothing has yielded to the scheduler. But I am surprised to learn that
await Task.Yield();
doesn't suffice. Somewhat ironically (but perfectly understandably, given that it involves no asynchronous execution) neither does
await task;
On the other hand,
await Task.Delay(1);
does seem to be enough, but I'm not clear on whether that's a guarantee or an accident of timing.
To ask the question one more way: is there any (reasonable) code I can write which will guarantee that all continuations for task have run before proceeding?

Can I guarantee runnable task continuations have been run?
By awaiting them.
var i = 0;
var taskCompletionSource = new TaskCompletionSource<object>();
var task = taskCompletionSource.Task;
var continuation = Task.Run(async () =>
{
await task;
i = 1;
});
// Synchronously complete `task`
taskCompletionSource.SetResult(null);
// wait for the continuation
await continuation;
// ouputs 1
Console.WriteLine(i);
That works if you're withing an asynchronous method. If you're not, make it asynchronous. You can technically block, too, (.Wait() instead of await) but that invites deadlocks, so be careful.

Assuming that this code runs in the context of an async method
var i = 0;
await Task.Run(async () =>
{
await task;
i = 1;
});
Console.WriteLine(i);
or
var i = 0;
await task;
i = 1;
Console.WriteLine(i);

Task.WhenAny blocking

I'm not sure if I'm misunderstanding the usage of Task.WhenAny but in the following code only "0" gets printed when it should print "1" and "2" and then "mx.Name" every time a task finishes:
public async void PopulateMxRecords(List<string> nsRecords, int threads)
{
ThreadPool.SetMinThreads(threads, threads);
var resolver = new DnsStubResolver();
var tasks = nsRecords.Select(ns => resolver.ResolveAsync<MxRecord>(ns, RecordType.Mx));
Console.WriteLine("0");
var finished = Task.WhenAny(tasks);
Console.WriteLine("1");
while (mxNsRecords.Count < nsRecords.Count)
{
Console.WriteLine("2");
var task = await finished;
var mxRecords = await task;
foreach(var mx in mxRecords)
Console.WriteLine(mx.Name);
}
}
The DnsStubResolver is part of ARSoft.Tools.Net.Dns. The nsRecords list contains up to 2 million strings.

I'm not sure if I'm misunderstanding the usage of Task.WhenAny
You might be. The pattern you seem to be looking for is interleaving. In the following example, notice the important changes that I have made:
use ToList() to materialize the LINQ query results,
move WhenAny() into the loop,
use Remove(task) as each task completes, and
run the while loop as long as tasks.Count() > 0.
Those are the important changes. The other changes are there to make your listing into a runnable demo of interleaving, the full listing of which is here: https://dotnetfiddle.net/nr1gQ7
public static async Task PopulateMxRecords(List<string> nsRecords)
{
var tasks = nsRecords.Select(ns => ResolveAsync(ns)).ToList();
while (tasks.Count() > 0)
{
var task = await Task.WhenAny(tasks);
tasks.Remove(task);
var mxRecords = await task;
Console.WriteLine(mxRecords);
}
}

creating a .net async wrapper to a sync request

I have the following situation (or a basic misunderstanding with the async await mechanism).
Assume you have a set of 1-20 web request call that takes a long time: findItemsByProduct().
you want to wrap it around in an async request, that would be able to abstract all these calls into one async call, but I can't seem to be able to do it without using more threads.
If I'm doing:
int total = result.paginationOutput.totalPages;
for (int i = 2; i < total + 1; i++)
{
await Task.Factory.StartNew(() =>
{
result = client.findItemsByProduct(i);
});
newList.AddRange(result.searchResult.item);
}
}
return newList;
problem here, that the calls don't run together, rather they are waiting one by one.
I would like all the calls to run together and than harvest the results.
as pseudo code, I would like the code to run like this:
forEach item {
result = item.makeWebRequest();
}
foreach item {
List.addRange(item.harvestResults);
}
I have no idea how to make the code to do that though..

Ideally, you should add a findItemsByProductAsync that returns a Task<Item[]>. That way, you don't have to create unnecessary tasks using StartNew or Task.Run.
Then your code can look like this:
int total = result.paginationOutput.totalPages;
// Start all downloads; each download is represented by a task.
Task<Item[]>[] tasks = Enumerable.Range(2, total - 1)
.Select(i => client.findItemsByProductAsync(i)).ToArray();
// Wait for all downloads to complete.
Item[][] results = await Task.WhenAll(tasks);
// Flatten the results into a single collection.
return results.SelectMany(x => x).ToArray();

Given your requirements which I see as:
Process n number of non-blocking tasks
Process results after all queries have returned
I would use the CountdownEvent for this e.g.
var results = new ConcurrentBag<ItemType>(result.pagination.totalPages);
using (var e = new CountdownEvent(result.pagination.totalPages))
{
for (int i = 2; i <= result.pagination.totalPages+1; i++)
{
Task.Factory.StartNew(() => return client.findItemsByProduct(i))
.ContinueWith(items => {
results.AddRange(items);
e.Signal(); // signal task is done
});
}
// Wait for all requests to complete
e.Wait();
}
// Process results
foreach (var item in results)
{
...
}

This particular problem is solved easily enough without even using await. Simply create each of the tasks, put all of the tasks into a list, and then use WhenAll on that list to get a task that represents the completion of all of those tasks:
public static Task<Item[]> Foo()
{
int total = result.paginationOutput.totalPages;
var tasks = new List<Task<Item>>();
for (int i = 2; i < total + 1; i++)
{
tasks.Add(Task.Factory.StartNew(() => client.findItemsByProduct(i)));
}
return Task.WhenAll(tasks);
}
Also note you have a major problem in how you use result in your code. You're having each of the different tasks all using the same variable, so there are race conditions as to whether or not it works properly. You could end up adding the same call twice and having one skipped entirely. Instead you should have the call to findItemsByProduct be the result of the task, and use that task's Result.

If you want to use async-await properly you have to declare your functions async, and the functions that call you also have to be async. This continues until you have once synchronous function that starts the async process.
Your function would look like this:
by the way you didn't describe what's in the list. I assume they are
object of type T. in that case result.SearchResult.Item returns
IEnumerable
private async Task<List<T>> FindItems(...)
{
int total = result.paginationOutput.totalPages;
var newList = new List<T>();
for (int i = 2; i < total + 1; i++)
{
IEnumerable<T> result = await Task.Factory.StartNew(() =>
{
return client.findItemsByProduct(i);
});
newList.AddRange(result.searchResult.item);
}
return newList;
}
If you do it this way, your function will be asynchronous, but the findItemsByProduct will be executed one after another. If you want to execute them simultaneously you should not await for the result, but start the next task before the previous one is finished. Once all tasks are started wait until all are finished. Like this:
private async Task<List<T>> FindItems(...)
{
int total = result.paginationOutput.totalPages;
var tasks= new List<Task<IEnumerable<T>>>();
// start all tasks. don't wait for the result yet
for (int i = 2; i < total + 1; i++)
{
Task<IEnumerable<T>> task = Task.Factory.StartNew(() =>
{
return client.findItemsByProduct(i);
});
tasks.Add(task);
}
// now that all tasks are started, wait until all are finished
await Task.WhenAll(tasks);
// the result of each task is now in task.Result
// the type of result is IEnumerable<T>
// put all into one big list using some linq:
return tasks.SelectMany ( task => task.Result.SearchResult.Item)
.ToList();
// if you're not familiar to linq yet, use a foreach:
var newList = new List<T>();
foreach (var task in tasks)
{
newList.AddRange(task.Result.searchResult.item);
}
return newList;
}