I have a MainMethod which needs to call two methods Method1 and Method2 parallel. Both of them will return list of Employee but from different database. I need to call them parallel and then combine the results of Method1 and Method2 in MainMethod and then return result to the caller of MainMethod.
I greatly appreciate if people can tell what must be signatures of methods and what code I need to write I mean async/await keywords.
Using a bit more shorthand...
public static async Task<IEnumerable<Employee>> MainMethod()
{
// Await when all to get an array of result sets after all of then have finished
var results = await Task.WhenAll(
Task.Run(() => Method1()), // Note that this leaves room for parameters to Method1...
Task.Run(Method2) // While this shorthands if there are no parameters
// Any further method calls can go here as more Task.Run calls
);
// Simply select many over the result sets to get each result
return results.SelectMany(r => r);
}
for signature reference, this uses the following .NET functions:
Task.WhenAll
Task.Run
Enumerable.SelectMany (as an extension method)
You can run them as 2 Task<T>s. The Result property takes care of the waiting. Approximately:
// untested
Task<List<Employee>> t1 = Task.Factory.StartNew(() => Method1());
Task<List<Employee>> t2 = Task.Factory.StartNew(() => Method2());
var result = t1.Result.Concat(t2.Result);
Related
I have a method in a class defined as....
public static async Task<int> GetDCountAsync(int dId)
I'm trying to call this method within a LINQ Where clause....
var excessD= dToConsider
.Where(async x => await myService.GetDCountAsync(x.Id) >= x.Threshold)
.Select(x => x.Id)
.ToArray();
FYI: Threshold is an int.
I'm getting an error for async....
The return type of an 'async' anonymous function must be a 'void', 'Task', 'Task', ......
The return type of GetDCountAsync is async Task. Where am I going wrong? Thanks in advance!
You could use the functionality already available in the System.Linq.Async package:
int[] excessD = await dToConsider
.ToAsyncEnumerable()
.WhereAwait(async x => await myService.GetDCountAsync(x.Id) >= x.Threshold)
.Select(x => x.Id)
.ToArrayAsync();
The signature of the WhereAwait operator:
// Filters the elements of an async-enumerable sequence based on
// an asynchronous predicate.
public static IAsyncEnumerable<TSource> WhereAwait<TSource>(
this IAsyncEnumerable<TSource> source,
Func<TSource, ValueTask<bool>> predicate);
The asynchronous method myService.GetDCountAsync will be invoked and awaited for one element at a time, not for all of them concurrently.
So the full error message you are getting will be :
Cannot convert async lambda expression to delegate type 'Func<int, bool>'. An async lambda expression may return void, Task or Task, none of which are convertible to 'Func<int, bool>'.
What it's actually telling you is that a .Where() LINQ call is expecting to take an int, and return a boolean. But because you are trying to do an async call, it's instead trying to return Task.
Or in other words, the compiler is saying "I see you are using an async method here, I know that async methods can only return void, Task or Task, and I also know that this Where method expects a bool back, so I know right now this is not gonna work"
Part of the problem is that we don't actually know what type dToConsider is, and in some cases, it's going to look to do deferred execution. In any case, because you are doing custom code logic for the WHERE statement, let's assume that this is not EntityFramework or something that needs to be deferred and say it's just a list. In that case, simple using a typical ForEach loop to filter your list down.
var filteredItems = new List<T>();
foreach(var item in dToConsider)
{
if(await myService.GetDCountAsync(x.Id) >= x.Threshold)
filteredItems.Add(item);
}
I'm sure someone may answer with a nice extension method for you using Task.WhenAll etc, but the this is the simplest way to get what you need done.
Basically I have a procedure like
var results = await Task.WhenAll(
from input in inputs
select Task.Run(async () => await InnerMethodAsync(input))
);
.
.
.
private static async Task<Output> InnerMethodAsync(Input input)
{
var x = await Foo(input);
var y = await Bar(x);
var z = await Baz(y);
return z;
}
and I'm wondering whether there's a fancy way to combine this into a single LINQ query that's like an "async stream" (best way I can describe it).
When you use LINQ, there are generally two parts to it: creation and iteration.
Creation:
var query = list.Select( a => a.Name);
These calls are always synchronous. But this code doesn't do much more than create an object that exposes an IEnumerable. The actual work isn't done till later, due to a pattern called deferred execution.
Iteration:
var results = query.ToList();
This code takes the enumerable and gets the value of each item, which typically will involve the invocation of your callback delegates (in this case, a => a.Name ). This is the part that is potentially expensive, and could benefit from asychronousness, e.g. if your callback is something like async a => await httpClient.GetByteArrayAsync(a).
So it's the iteration part that we're interested in, if we want to make it async.
The issue here is that ToList() (and most of the other methods that force iteration, like Any() or Last()) are not asynchronous methods, so your callback delegate will be invoked synchronously, and you’ll end up with a list of tasks instead of the data you want.
We can get around that with a piece of code like this:
public static class ExtensionMethods
{
static public async Task<List<T>> ToListAsync<T>(this IEnumerable<Task<T>> This)
{
var tasks = This.ToList(); //Force LINQ to iterate and create all the tasks. Tasks always start when created.
var results = new List<T>(); //Create a list to hold the results (not the tasks)
foreach (var item in tasks)
{
results.Add(await item); //Await the result for each task and add to results list
}
return results;
}
}
With this extension method, we can rewrite your code:
var results = await inputs.Select( async i => await InnerMethodAsync(i) ).ToListAsync();
^That should give you the async behavior you're looking for, and avoids creating thread pool tasks, as your example does.
Note: If you are using LINQ-to-entities, the expensive part (the data retrieval) isn't exposed to you. For LINQ-to-entities, you'd want to use the ToListAsync() that comes with the EF framework instead.
Try it out and see the timings in my demo on DotNetFiddle.
A rather obvious answer, but you have just used LINQ and async together - you're using LINQ's select to project, and start, a bunch of async Tasks, and then await on the results, which provides an asynchronous parallelism pattern.
Although you've likely just provided a sample, there are a couple of things to note in your code (I've switched to Lambda syntax, but the same principals apply)
Since there's basically zero CPU bound work on each Task before the first await (i.e. no work done before var x = await Foo(input);), there's no real reason to use Task.Run here.
And since there's no work to be done in the lambda after call to InnerMethodAsync, you don't need to wrap the InnerMethodAsync calls in an async lambda (but be wary of IDisposable)
i.e. You can just select the Task returned from InnerMethodAsync and await these with Task.WhenAll.
var tasks = inputs
.Select(input => InnerMethodAsync(input)) // or just .Select(InnerMethodAsync);
var results = await Task.WhenAll(tasks);
More complex patterns are possible with asynchronony and Linq, but rather than reinventing the wheel, you should have a look at Reactive Extensions, and the TPL Data Flow Library, which have many building blocks for complex flows.
Try using Microsoft's Reactive Framework. Then you can do this:
IObservable<Output[]> query =
from input in inputs.ToObservable()
from x in Observable.FromAsync(() => Foo(input))
from y in Observable.FromAsync(() => Bar(x))
from z in Observable.FromAsync(() => Baz(y))
select z;
Output[] results = await query.ToArray();
Simple.
Just NuGet "System.Reactive" and add using System.Reactive.Linq; to your code.
I am have the following code to run multiple async tasks and wait for all the results.
string[] personStoreNames = _faceStoreRepo.GetPersonStoreNames();
IEnumerable<Task<IdentifyResult[]>> identifyFaceTasks =
personStoreNames.Select(storename => _faceServiceClient.IdentifyAsync(storename, faceIds, 1));
var recognitionresults =
await Task.WhenAll(identifyFaceTasks);
When I get the results how can I get the storename for each task result. Each array of IdentifyResult will be for a certain storename, but I'm not sure how to end up with my IdentifyResults and the storename they were found in.
As MSDN says use same indexes to get results that you used for parameters.
WhenAll
If none of the tasks faulted and none of the tasks were canceled, the resulting task will end in the TaskStatus.RanToCompletion state. The Result of the returned task will be set to an array containing all of the results of the supplied tasks in the same order as they were provided (e.g. if the input tasks array contained t1, t2, t3, the output task's Result will return an TResult[] where arr[0] == t1.Result, arr[1] == t2.Result, and arr[2] == t3.Result).
This is not a direct answer to the question, but you can use Microsoft's Reactive Framework to make this code a bit neater.
You can write this:
var query =
from sn in _faceStoreRepo.GetPersonStoreNames().ToObservable()
from irs in Observable.FromAsync(() => _faceServiceClient.IdentifyAsync(sn, faceIds, 1))
select new { sn, irs };
var result = await query.ToArray();
result is an array of anonymous variables of new { sn, irs }.
One advantage is that you can process the values as they become available:
var result = await query
.Do(x => { /* process each `x.sn` & `x.irs` pair as they arrive */ })
.ToArray();
I would want to call an async method inside lambda expression. Please help me doing the below
eg -
return xyz.Where(async x=> await AsyncMethodCall(x.val));
And the Async method looks like
public async Task<bool> AsyncMethodCall(Data d){...}
When I do the above, I get the following error
Error CS4010 Cannot convert async lambda expression to delegate type
'Func<Data, bool>'. An async lambda expression may return void, Task
or Task<T>, none of which are convertible to 'Func<Data, bool>'.
Thanks in advance for the help
Asynchronous sequences are tricky because you really have to think about what specifically you want the code to do.
For example, you could want to execute the AsyncMethodCall calls sequentially, and then return all the results:
var result = new List<T>();
foreach (var d in xyz)
if (await AsyncMethodCall(d.val))
result.Add(d);
return result;
Or, you could execute all the AsyncMethodCall calls concurrently, and then collect and return the results (again, all at once):
var tasks = xyz.Select(async d => new { d, filter = await AsyncMethodCall(d.val) });
var results = await Task.WhenAll(tasks);
return results.Where(x => x.filter).Select(x => x.d);
Or, you could execute all the AsyncMethodCall calls sequentially, and produce the results one at a time. This approach is incompatible with IEnumerable<T> (assuming you want to keep the call asynchronous). If you want to produce a sequence where AsyncMethodCall is asynchronously invoked during sequence enumeration, then you would need to change to IAsyncEnumerable<T>. If you want to produce a sequence that is started by the consumer and then produces results on its own, you would need to change to IObservable<T>.
Or, you could execute all the AsyncMethodCall calls concurrently, and produce the results one at a time. This is also incompatible with IEnumerable<T>; you would need to change to IObservable<T>. And you would also need to decide whether to maintain the original ordering, or to produce them in order of AsyncMethodCall completing.
Here is my code which generates XmlNodes in async way and then inserts these nodes into main document - sequentially because it is fast process and I also may need to keep certain order.
There is about 15 nodes to import. How can I refactor this code so the code is more compact?
XmlNode soccerNode = null;
XmlNode basketbalNode = null;
XmlNode hockeyNode = null;
...
var tasks = new List<Task>
{
Task.Factory.StartNew(() => soccerNode = this.getSoccer()),
Task.Factory.StartNew(() => basketbalNode = this.getBasketball()),
Task.Factory.StartNew(() => hockeyNode = this.getHockey()),
...};
Task.WaitAll(tasks.ToArray());
AddToMainDocument(soccerNode);
AddToMainDocument(basketbalNode);
AddToMainDocument(hockeyNode);
...
I'm guessing getSoccer, getBasketball and getHockey are properties since your code example did not have any method parenthesis? If they are methods then just add the missing parenthesis to the code below.
var soccerTask = Task.Run(() => getSoccer).ConfigureAwait(false);
var basketbalTask = Task.Run(() => getBasketball).ConfigureAwait(false);
var hockeyTask = Task.Run(() => getHockey).ConfigureAwait(false);
AddToMainDocument(await soccerTask);
AddToMainDocument(await basketbalTask);
AddToMainDocument(await hockeyTask);
All three jobs are executed asynchronously and then you await them in the order you need them.
Regarding how to make your code more compact, I would need to know more about it. The methods/properties, are they all within the same object? Should all of them always be called? You could use reflection to find all of your methods, have attributes on them to specify order and use that information to spin up tasks and await them in the right order.
But to be honest, if you know which methods needs to be called then just do it manually as in your example. If you have a dynamic object where methods are added/removed then you should probably use reflection to do the job, otherwise it's just an unnecessary overhead.
A think this might help:
var t1 = Task.Factory.StartNew(() => soccerNode = /*Do something */);
var t2 = Task.Factory.StartNew(() => basketbalNode = /*Do something */);
var tasks = new List<Task> { t1, t2 };
Task.WaitAll(tasks.ToArray());
AddToMainDocument(t1.Result);
AddToMainDocument(t2.Result);
We wait until they are all done then add to the main document.