We Keep Coding

How to write async methods with await, but make them as short as with Task.Run?

After browsing different await vs Task.Run questions on SO my takeaway is that await is better for I/O operations and Task.Run for CPU-intensive operations. However the code with await seems to always be longer than Task.Run. Below is an example method of how I am creating a context object with await in my app:
public async AppContext CreateAppContext()
{
var context = new AppContext();
var customersTask = _dataAccess.GetCustomers();
var usersTask = _dataAccess.GetUsers();
var reportsTask = _dataAccess.GetReports();
context.Customers = await customersTask;
context.Users = await usersTask;
context.Reports = await reportsTask;
return context;
}
If I was to rewrite this with Task.Run I could do
public async AppContext CreateAppContext()
{
var context = new AppContext();
await Task.WhenAll(new[]
{
Task.Run(async () => { context.Customers = await _dataAccess.GetCustomers(); }),
Task.Run(async () => { context.Users = await _dataAccess.GetUsers(); }),
Task.Run(async () => { context.Reports = await _dataAccess.GetReports(); });
})
return context;
}
The difference is not major when I create an object with 3 properties but I have objects where I need to initialize 20+ properties in this manner which makes the await code a lot longer (nearly double) than Task.Run. Is there a way for me to initialize the object using await with code that is not a lot longer than what I can do with Task.Run?

Personally, I prefer an asynchronous factory pattern over that kind of code; but if you need to do concurrent asynchronous work like that multiple times, then you'll probably want to write your own helper method.
The BCL-provided WhenAll works best when either all tasks have no results, or when all tasks have the same type of result. One fairly common approach to help WhenAll work with different types of tasks is to return a tuple of results, which can then be deconstructed into different variables if desired. E.g.,
public static class TaskEx
{
public static async Task<(T1, T2, T3)> WhenAll<T1, T2, T3>(Task<T1> task1, Task<T2> task2, Task<T3> task3)
{
await Task.WhenAll(task1, task2, task3);
return (await task1, await task2, await task3);
}
}
// Usage
public async AppContext CreateAppContext()
{
var context = new AppContext();
(context.Customers, context.Users, conext.Reports) =
await TaskEx.WhenAll(
_dataAccess.GetCustomers(),
_dataAccess.GetUsers(),
_dataAccess.GetReports());
return context;
}
You can even define a tuple GetAwaiter extension method if you want, to make it more implicit:
// Usage
public async AppContext CreateAppContext()
{
var context = new AppContext();
(context.Customers, context.Users, conext.Reports) =
await (
_dataAccess.GetCustomers(),
_dataAccess.GetUsers(),
_dataAccess.GetReports());
return context;
}
There are a couple of disadvantages to these approaches, though. First, you have to define as many overloads as you need. Second, the multiple-assignment code is not very nice; it's fine for 2 or 3 properties, but would get ugly (IMO) if done for much more than that.
So I think what you really want is a custom delegate form of WhenAll. Something like this should work:
public static class TaskEx
{
public static async Task WhenAll(params Func<Task>[] tasks)
{
return Task.WhenAll(tasks.Select(action => action()));
}
}
// Usage
public async AppContext CreateAppContext()
{
var context = new AppContext();
await TaskEx.WhenAll(
async () => context.Customers = await _dataAccess.GetCustomers(),
async () => context.Users = await _dataAccess.GetUsers(),
async () => conext.Reports = await _dataAccess.GetReports());
return context;
}
Since this solution avoids dealing with the different result types entirely, multiple overloads aren't needed.

If you really want to keep your general pattern (which I'd avoid - it would be much better to do all the work and then assign all the results at the same time; look into the return value of Task.WhenAll), all you need is a simple helper method:
static async Task Assign<T>(Action<T> assignment, Func<Task<T>> getValue)
=> assignment(await getValue());
Then you can use it like this:
await Task.WhenAll
(
Assign(i => context.Customers = i, _dataAccess.GetCustomers),
Assign(i => context.Users = i, _dataAccess.GetUsers),
Assign(i => context.Reports = i, _dataAccess.GetReports)
);
There's many other ways to make this even simpler, but this is the most equivalent to your Task.Run code without having to involve another thread indirection just to do an assignment. It also avoids the very common mistake when you happen to use the wrong Task.Run overload and get a race condition (as Task.Run returns immediately instead of waiting for the result).
Also, you misunderstood the "await vs. Task.Run" thing. There's actually not that much difference between await and Task.Run in your code - mainly, it forces a thread switch (and a few other subtle things). The argument is against using Task.Run to run synchronous code; that wastes a thread waiting for a thing to complete, your code doesn't.
Do keep in mind that WhenAll comes with its own complications, though. While it does mean you don't have to worry about some of the tasks ending up unobserved (and not waited on!), it also means you have to completely rethink your exception handling, since you're going to get an AggregateException rather than anything more specific. If you're relying on error handling based on identifying exceptions, you need to be very careful. Usually, you don't want AggregateException to leak out of methods - it's very difficult to handle in a global manner; the only method that knows the possibilities of what can happen is the method that calls the WhenAll. Hopefully.
It's definitely a good idea to run parallel operations like this in a way that cannot produce dangerous and confusing side-effects. In your code, you either get a consistent object returned, or you get nothing - that's exactly the right approach. Be wary of this approach leaking into other contexts - it can get really hard to debug issues where randomly half of the operations succeed and other half fails :)

Is there a way for me to initialize the object using await with code that is not a lot longer than what I can do with Task.Run?
If you want to run all tasks in parallel - in short no, you cant shorten number of lines. Also note that those two snippets are not fully functionally equivalent - see Why should I prefer single await Task.WhenAll over multiple awaits.
You can simplify (and maybe even improve performance a bit) your Task.WhenAll approach by introducing a method which will await and assign. Something along these lines:
public async AppContext CreateAppContext()
{
var context = new AppContext();
await Task.WhenAll(
AwaitAndAssign(val => context.Customers = val, _dataAccess.GetCustomers()),
AwaitAndAssign(val => context.Users = val, _dataAccess.Users()),
AwaitAndAssign(val => context.Reports = val, _dataAccess.GetReports())
);
return context;
async Task AwaitAndAssign<T>(Action<T> assign, Task<T> valueTask) =>
assign(await valueTask);
}

C# Add to a List Asynchronously in API

I have an API which needs to be run in a loop for Mass processing.
Current single API is:
public async Task<ActionResult<CombinedAddressResponse>> GetCombinedAddress(AddressRequestDto request)
We are not allowed to touch/modify the original single API. However can be run in bulk, using foreach statement. What is the best way to run this asychronously without locks?
Current Solution below is just providing a list, would this be it?
public async Task<ActionResult<List<CombinedAddressResponse>>> GetCombinedAddress(List<AddressRequestDto> requests)
{
var combinedAddressResponses = new List<CombinedAddressResponse>();
foreach(AddressRequestDto request in requests)
{
var newCombinedAddress = (await GetCombinedAddress(request)).Value;
combinedAddressResponses.Add(newCombinedAddress);
}
return combinedAddressResponses;
}
Update:
In debugger, it has to go to combinedAddressResponse.Result.Value
combinedAddressResponse.Value = null
and Also strangely, writing combinedAddressResponse.Result.Value gives error below "Action Result does not contain a definition for for 'Value' and no accessible extension method

I'm writing this code off the top of my head without an IDE or sleep, so please comment if I'm missing something or there's a better way.
But effectively I think you want to run all your requests at once (not sequentially) doing something like this:
public async Task<ActionResult<List<CombinedAddressResponse>>> GetCombinedAddress(List<AddressRequestDto> requests)
{
var combinedAddressResponses = new List<CombinedAddressResponse>(requests.Count);
var tasks = new List<Task<ActionResult<CombinedAddressResponse>>(requests.Count);
foreach (var request in requests)
{
tasks.Add(Task.Run(async () => await GetCombinedAddress(request));
}
//This waits for all the tasks to complete
await tasks.WhenAll(tasks.ToArray());
combinedAddressResponses.AddRange(tasks.Select(x => x.Result.Value));
return combinedAddressResponses;
}

looking for a way to speed things up and run in parallel thanks
What you need is "asynchronous concurrency". I use the term "concurrency" to mean "doing more than one thing at a time", and "parallel" to mean "doing more than one thing at a time using threads". Since you're on ASP.NET, you don't want to use additional threads; you'd want to use a form of concurrency that works asynchronously (which uses fewer threads). So, Parallel and Task.Run should not be parts of your solution.
The way to do asynchronous concurrency is to build a collection of tasks, and then use await Task.WhenAll. E.g.:
public async Task<ActionResult<IReadOnlyList<CombinedAddressResponse>>> GetCombinedAddress(List<AddressRequestDto> requests)
{
// Build the collection of tasks by doing an asynchronous operation for each request.
var tasks = requests.Select(async request =>
{
var combinedAddressResponse = await GetCombinedAdress(request);
return combinedAddressResponse.Value;
}).ToList();
// Wait for all the tasks to complete and get the results.
var results = await Task.WhenAll(tasks);
return results;
}

How does parallelization work on async/await?

I have the following code, that I intend to run asynchronously. My goal is that GetPictureForEmployeeAsync() is called in parallel as many times as needed. I'd like to make sure that 'await' on CreatePicture does not prevent me from doing so.
public Task<Picture[]> GetPictures(IDictionary<string, string> tags)
{
var query = documentRepository.GetRepositoryQuery();
var employees = query.Where(doc => doc.Gender == tags["gender"]);
return Task.WhenAll(employees.Select(employee => GetPictureForEmployeeAsync(employee, tags)));
}
private Task<Picture> GetPictureForEmployeeAsync(Employee employee, IDictionary<string, string> tags)
{
var base64PictureTask = blobRepository.GetBase64PictureAsync(employee.ID.ToString());
var documentTask = documentRepository.GetItemAsync(employee.ID.ToString());
return CreatePicture(tags, base64PictureTask, documentTask);
}
private static async Task<Picture> CreatePicture(IDictionary<string, string> tags, Task<string> base64PictureTask, Task<Employee> documentTask)
{
var document = await documentTask;
return new Picture
{
EmployeeID = document.ID,
Data = await base64PictureTask,
ID = document.ID.ToString(),
Tags = tags,
};
}
If I understand it correctly, Task.WhenAll() is not affected by the two awaited tasks inside CreatePicture() because GetPictureForEmployeeAsync() is not awaited. Am I right about this? If not, how should I restructure the code to achieve what I want?

I'd like to make sure that 'await' on CreatePicture does not prevent me from doing so.
It doesn't.
If I understand it correctly, Task.WhenAll() is not affected by the two awaited tasks inside CreatePicture() because GetPictureForEmployeeAsync() is not awaited. Am I right about this?
Yes and no. The WhenAll isn't limited in any way by the awaited tasks in CreatePicture, but that has nothing to do with whether GetPictureForEmployeeAsync is awaited or not. These two lines of code are equivalent in terms of behavior:
return Task.WhenAll(employees.Select(employee => GetPictureForEmployeeAsync(employee, tags)));
return Task.WhenAll(employees.Select(async employee => await GetPictureForEmployeeAsync(employee, tags)));
I recommend reading my async intro to get a good understanding of how async and await work with tasks.
Also, since GetPictures has non-trivial logic (GetRepositoryQuery and evaluating tags["gender"]), I recommend using async and await for GetPictures, as such:
public async Task<Picture[]> GetPictures(IDictionary<string, string> tags)
{
var query = documentRepository.GetRepositoryQuery();
var employees = query.Where(doc => doc.Gender == tags["gender"]);
var tasks = employees.Select(employee => GetPictureForEmployeeAsync(employee, tags)).ToList();
return await Task.WhenAll(tasks);
}
As a final note, you may find your code cleaner if you don't pass around "tasks meant to be awaited" - instead, await them first and pass their result values:
async Task<Picture> GetPictureForEmployeeAsync(Employee employee, IDictionary<string, string> tags)
{
var base64PictureTask = blobRepository.GetBase64PictureAsync(employee.ID.ToString());
var documentTask = documentRepository.GetItemAsync(employee.ID.ToString());
await Task.WhenAll(base64PictureTask, documentTask);
return CreatePicture(tags, await base64PictureTask, await documentTask);
}
static Picture CreatePicture(IDictionary<string, string> tags, string base64Picture, Employee document)
{
return new Picture
{
EmployeeID = document.ID,
Data = base64Picture,
ID = document.ID.ToString(),
Tags = tags,
};
}

The thing to keep in mind about calling an async method is that, as soon as an await statement is reached inside that method, control immediately goes back to the code that invoked the async method -- no matter where the await statement happens to be in the method. With a 'normal' method, control doesn't go back to the code that invokes that method until the end of that method is reached.
So in your case, you can do the following:
private async Task<Picture> GetPictureForEmployeeAsync(Employee employee, IDictionary<string, string> tags)
{
// As soon as we get here, control immediately goes back to the GetPictures
// method -- no need to store the task in a variable and await it within
// CreatePicture as you were doing
var picture = await blobRepository.GetBase64PictureAsync(employee.ID.ToString());
var document = await documentRepository.GetItemAsync(employee.ID.ToString());
return CreatePicture(tags, picture, document);
}
Because the first line of code in GetPictureForEmployeeAsync has an await, control will immediately go right back to this line...
return Task.WhenAll(employees.Select(employee => GetPictureForEmployeeAsync(employee, tags)));
...as soon as it is invoked. This will have the effect of all of the employee items getting processed in parallel (well, sort of -- the number of threads that will be allotted to your application will be limited).
As an additional word of advice, if this application is hitting a database or web service to get the pictures or documents, this code will likely cause you issues with running out of available connections. If this is the case, consider using System.Threading.Tasks.Parallel and setting the maximum degree of parallelism, or use SemaphoreSlim to control the number of connections used simultaneously.

How to convert WaitAll() to async and await

Is it possible to rewrite the code below using the async await syntax?
private void PullTablePages()
{
var taskList = new List<Task>();
var faultedList = new List<Task>();
foreach (var featureItem in featuresWithDataName)
{
var task = Task.Run(() => PullTablePage();
taskList.Add(task);
if(taskList.Count == Constants.THREADS)
{
var index = Task.WaitAny(taskList.ToArray());
taskList.Remove(taskList[index]);
}
}
if (taskList.Any())
{
Task.WaitAll(taskList.ToArray());
}
//todo: do something with faulted list
}
When I rewrote it as below, the code doesn't block and the console application finishes before most of the threads complete.
It seems like the await syntax doesn't block as I expected.
private async void PullTablePagesAsync()
{
var taskList = new List<Task>();
var faultedList = new List<Task>();
foreach (var featureItem in featuresWithDataName)
{
var task = Task.Run(() => PullTablePage();
taskList.Add(task);
if(taskList.Count == Constants.THREADS)
{
var anyFinished = await Task.WhenAny(taskList.ToArray());
await anyFinished;
for (var index = taskList.Count - 1; index >= 0; index--)
{
if (taskList[index].IsCompleted)
{
taskList.Remove(taskList[index]);
}
}
}
}
if (taskList.Any())
{
await Task.WhenAll(taskList);
}
//todo: what to do with faulted list?
}
Is it possible to do so?
WaitAll doesn't seem to wait for all tasks to complete. How do I get it to do so? The return type says that it returns a task, but can't seem to figure out the syntax.## Heading ##
New to multithreading, please excuse ignorance.

Is it possible to rewrite the code below using the async await syntax?
Yes and no. Yes, because you already did it. No, because while being equivalent from the function implementation standpoint, it's not equivalent from the function caller perspective (as you already experienced with your console application). Contrary to many other C# keywords, await is not a syntax sugar. There is a reason why the compiler forces you to mark your function with async in order to enable await construct, and the reason is that now your function is no more blocking and that puts additional responsibility to the callers - either put themselves to be non blocking (async) or use a blocking calls to your function. Every async method in fact is and should return Task or Task<TResult> and then the compiler will warn the callers that ignore that fact. The only exception is async void which is supposed to be used only for event handlers, which by nature should not care what the object being notified is doing.
Shortly, async/await is not for rewriting synchronous (blocking code), but for easy turning it to asynchronous (non blocking). If your function is supposed to be synchronous, then just keep it the way it is (and your original implementation is perfectly doing that). But if you need asynchronous version, then you should change the signature to
private async Task PullTablePagesAsync()
with the await rewritten body (which you already did correctly). And for backward compatibility provide the old synchronous version using the new implementation like this
private void PullTablePages() { PullTablePagesAsync().Wait(); }

It seems like the await syntax doesn't block as I expected.
You're expecting the wrong thing.
The await syntax should never block - it's just that the execution flow should not continue until the task is finished.
Usually you are using async/await in methods that return a task. In your case you're using it in a method with a return type void.
It takes times to get your head around async void methods, that's why their use is usually discouraged. Async void methods run synchronously (block) to the calling method until the first (not completed) task is awaited. What happens after depends on the execution context (usually you're running on the pool). What's important: The calling method (the one that calls PullTablePAgesAsync) does not know of continuations and can't know when all code in PullTablePagesAsync is done.
Maybe take a look on the async/await best practices on MSDN

C# async method execute base on the first completed task

I have a program that will read some values from 2 readers located at different physical locations. For one reader I know I can write like:
private async void waitForReading()
{
string result = await readFromReader1();
return result;
}
private async Task<string> readFromReader1()
{
//poll until value detected
return reader1Value;
}
private async Task<string> readFromReader2()
{
//poll until value detected
return reader2Value;
}
However, what if I read from two readers and resume execution when one of the task returned?
What I want to achieve looks like:
private async void waitForReading()
{
string result = await readFromReader1() || readFromReader2();
return result;
}
Is it possible?

The basic answer to your question is to use WhenAny:
var task1 = readFromReader1Async();
var task2 = readFromReader2Async();
return await await Task.WhenAny(task1, task2);
However, there are a few additional considerations.
You mention "polling" in your implementation. This is the kind of operation that you'd probably want to cancel if the other reader has already returned a value. Consider using CancellationToken to enable cancellation.
Think about how to model the data coming from your "reader devices". If you always just want to query them on demand, then an async approach is fine. However, if their data can change at any time (which I suspect is the case), then you might want to consider setting up a permanent poll and consume it as an event stream; in this case, Reactive Extensions would be a more appropriate model.

How about Task.WaitAny
Task<String> task1 = readFromReader1();
Task<String> task2 = readFromReader2();
String result = Task.WhenAny(new Task[]{task1, task2}).Result;

By looking into a similar question:
Run two async tasks in parallel and collect results in .NET 4.5
I get inspired by Task.WhenAll method posted by bart, and find that Task.WhenAny(Task[]) is what I need.