We Keep Coding

Why dont receive data if i have await [duplicate]

Is it possible to use Async when using ForEach? Below is the code I am trying:
using (DataContext db = new DataLayer.DataContext())
{
db.Groups.ToList().ForEach(i => async {
await GetAdminsFromGroup(i.Gid);
});
}
I am getting the error:
The name 'Async' does not exist in the current context
The method the using statement is enclosed in is set to async.

List<T>.ForEach doesn't play particularly well with async (neither does LINQ-to-objects, for the same reasons).
In this case, I recommend projecting each element into an asynchronous operation, and you can then (asynchronously) wait for them all to complete.
using (DataContext db = new DataLayer.DataContext())
{
var tasks = db.Groups.ToList().Select(i => GetAdminsFromGroupAsync(i.Gid));
var results = await Task.WhenAll(tasks);
}
The benefits of this approach over giving an async delegate to ForEach are:
Error handling is more proper. Exceptions from async void cannot be caught with catch; this approach will propagate exceptions at the await Task.WhenAll line, allowing natural exception handling.
You know that the tasks are complete at the end of this method, since it does an await Task.WhenAll. If you use async void, you cannot easily tell when the operations have completed.
This approach has a natural syntax for retrieving the results. GetAdminsFromGroupAsync sounds like it's an operation that produces a result (the admins), and such code is more natural if such operations can return their results rather than setting a value as a side effect.

This little extension method should give you exception-safe async iteration:
public static async Task ForEachAsync<T>(this List<T> list, Func<T, Task> func)
{
foreach (var value in list)
{
await func(value);
}
}
Since we're changing the return type of the lambda from void to Task, exceptions will propagate up correctly. This will allow you to write something like this in practice:
await db.Groups.ToList().ForEachAsync(async i => {
await GetAdminsFromGroup(i.Gid);
});

Starting with C# 8.0, you can create and consume streams asynchronously.
private async void button1_Click(object sender, EventArgs e)
{
IAsyncEnumerable<int> enumerable = GenerateSequence();
await foreach (var i in enumerable)
{
Debug.WriteLine(i);
}
}
public static async IAsyncEnumerable<int> GenerateSequence()
{
for (int i = 0; i < 20; i++)
{
await Task.Delay(100);
yield return i;
}
}
More

The simple answer is to use the foreach keyword instead of the ForEach() method of List().
using (DataContext db = new DataLayer.DataContext())
{
foreach(var i in db.Groups)
{
await GetAdminsFromGroup(i.Gid);
}
}

Here is an actual working version of the above async foreach variants with sequential processing:
public static async Task ForEachAsync<T>(this List<T> enumerable, Action<T> action)
{
foreach (var item in enumerable)
await Task.Run(() => { action(item); }).ConfigureAwait(false);
}
Here is the implementation:
public async void SequentialAsync()
{
var list = new List<Action>();
Action action1 = () => {
//do stuff 1
};
Action action2 = () => {
//do stuff 2
};
list.Add(action1);
list.Add(action2);
await list.ForEachAsync();
}
What's the key difference? .ConfigureAwait(false); which keeps the context of main thread while async sequential processing of each task.

This is not an old question, but .Net 6 introduced Parallel.ForeachAsync:
var collectionToIterate = db.Groups.ToList();
await Parallel.ForEachAsync(collectionToIterate, async (i, token) =>
{
await GetAdminsFromGroup(i);
});
ForeachAsync also accepts a ParallelOptions object, but usually you don't want to mess with the MaxDegreeOfParallelism property:
ParallelOptions parallelOptions = new ParallelOptions { MaxDegreeOfParallelism = 4 };
var collectionToIterate = db.Groups.ToList();
await Parallel.ForEachAsync(collectionToIterate, parallelOptions , async (i, token) =>
{
await GetAdminsFromGroup(i);
});
From Microsoft Docs: https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.paralleloptions.maxdegreeofparallelism?view=net-6.0
By default, For and ForEach will utilize however many threads the underlying scheduler provides, so changing MaxDegreeOfParallelism from the default only limits how many concurrent tasks will be used.
Generally, you do not need to modify this setting....

Add this extension method
public static class ForEachAsyncExtension
{
public static Task ForEachAsync<T>(this IEnumerable<T> source, int dop, Func<T, Task> body)
{
return Task.WhenAll(from partition in Partitioner.Create(source).GetPartitions(dop)
select Task.Run(async delegate
{
using (partition)
while (partition.MoveNext())
await body(partition.Current).ConfigureAwait(false);
}));
}
}
And then use like so:
Task.Run(async () =>
{
var s3 = new AmazonS3Client(Config.Instance.Aws.Credentials, Config.Instance.Aws.RegionEndpoint);
var buckets = await s3.ListBucketsAsync();
foreach (var s3Bucket in buckets.Buckets)
{
if (s3Bucket.BucketName.StartsWith("mybucket-"))
{
log.Information("Bucket => {BucketName}", s3Bucket.BucketName);
ListObjectsResponse objects;
try
{
objects = await s3.ListObjectsAsync(s3Bucket.BucketName);
}
catch
{
log.Error("Error getting objects. Bucket => {BucketName}", s3Bucket.BucketName);
continue;
}
// ForEachAsync (4 is how many tasks you want to run in parallel)
await objects.S3Objects.ForEachAsync(4, async s3Object =>
{
try
{
log.Information("Bucket => {BucketName} => {Key}", s3Bucket.BucketName, s3Object.Key);
await s3.DeleteObjectAsync(s3Bucket.BucketName, s3Object.Key);
}
catch
{
log.Error("Error deleting bucket {BucketName} object {Key}", s3Bucket.BucketName, s3Object.Key);
}
});
try
{
await s3.DeleteBucketAsync(s3Bucket.BucketName);
}
catch
{
log.Error("Error deleting bucket {BucketName}", s3Bucket.BucketName);
}
}
}
}).Wait();

If you are using EntityFramework.Core there is an extension method ForEachAsync.
The example usage looks like this:
using Microsoft.EntityFrameworkCore;
using System.Threading.Tasks;
public class Example
{
private readonly DbContext _dbContext;
public Example(DbContext dbContext)
{
_dbContext = dbContext;
}
public async void LogicMethod()
{
await _dbContext.Set<dbTable>().ForEachAsync(async x =>
{
//logic
await AsyncTask(x);
});
}
public async Task<bool> AsyncTask(object x)
{
//other logic
return await Task.FromResult<bool>(true);
}
}

I would like to add that there is a Parallel class with ForEach function built in that can be used for this purpose.

The problem was that the async keyword needs to appear before the lambda, not before the body:
db.Groups.ToList().ForEach(async (i) => {
await GetAdminsFromGroup(i.Gid);
});

This is method I created to handle async scenarios with ForEach.
If one of tasks fails then other tasks will continue their execution.
You have ability to add function that will be executed on every exception.
Exceptions are being collected as aggregateException at the end and are available for you.
Can handle CancellationToken
public static class ParallelExecutor
{
/// <summary>
/// Executes asynchronously given function on all elements of given enumerable with task count restriction.
/// Executor will continue starting new tasks even if one of the tasks throws. If at least one of the tasks throwed exception then <see cref="AggregateException"/> is throwed at the end of the method run.
/// </summary>
/// <typeparam name="T">Type of elements in enumerable</typeparam>
/// <param name="maxTaskCount">The maximum task count.</param>
/// <param name="enumerable">The enumerable.</param>
/// <param name="asyncFunc">asynchronous function that will be executed on every element of the enumerable. MUST be thread safe.</param>
/// <param name="onException">Acton that will be executed on every exception that would be thrown by asyncFunc. CAN be thread unsafe.</param>
/// <param name="cancellationToken">The cancellation token.</param>
public static async Task ForEachAsync<T>(int maxTaskCount, IEnumerable<T> enumerable, Func<T, Task> asyncFunc, Action<Exception> onException = null, CancellationToken cancellationToken = default)
{
using var semaphore = new SemaphoreSlim(initialCount: maxTaskCount, maxCount: maxTaskCount);
// This `lockObject` is used only in `catch { }` block.
object lockObject = new object();
var exceptions = new List<Exception>();
var tasks = new Task[enumerable.Count()];
int i = 0;
try
{
foreach (var t in enumerable)
{
await semaphore.WaitAsync(cancellationToken);
tasks[i++] = Task.Run(
async () =>
{
try
{
await asyncFunc(t);
}
catch (Exception e)
{
if (onException != null)
{
lock (lockObject)
{
onException.Invoke(e);
}
}
// This exception will be swallowed here but it will be collected at the end of ForEachAsync method in order to generate AggregateException.
throw;
}
finally
{
semaphore.Release();
}
}, cancellationToken);
if (cancellationToken.IsCancellationRequested)
{
break;
}
}
}
catch (OperationCanceledException e)
{
exceptions.Add(e);
}
foreach (var t in tasks)
{
if (cancellationToken.IsCancellationRequested)
{
break;
}
// Exception handling in this case is actually pretty fast.
// https://gist.github.com/shoter/d943500eda37c7d99461ce3dace42141
try
{
await t;
}
#pragma warning disable CA1031 // Do not catch general exception types - we want to throw that exception later as aggregate exception. Nothing wrong here.
catch (Exception e)
#pragma warning restore CA1031 // Do not catch general exception types
{
exceptions.Add(e);
}
}
if (exceptions.Any())
{
throw new AggregateException(exceptions);
}
}
}

How to pass additional context information along with tasks to Task.WhenAll?

Consider the following ideal code (which doesn't work). I'm essentially trying to create a list of Tasks that return a specific object, associate them with a string identifier, then execute all of them in bulk with Task.WhenAll. At the end of the execution, I need to have the results of those Tasks still associated with the string identifiers they were originally created with:
public async Task<SomeObject> DoSomethingAsync(string thing)
{
// implementation elided
}
public async Task<SomeObject> DoSomethingElseAsync(string thing)
{
// different implementation elided
}
public async Task<IEnumerable<(string, SomeObject)>>
DoManyThingsAsync(IEnumerable<string> someListOfStrings, bool condition)
{
var tasks = new List<(string, Task<SomeObject>)>();
foreach (var item in someListOfStrings)
{
if (condition)
{
tasks.Add((item, DoSomethingAsync(item)));
}
else
{
tasks.Add((item, DoSomethingElseAsync(item)));
}
}
// this doesn't compile, i'm just demonstrating what i want to achieve
var results = await Task.WhenAll(tasks);
return results;
}
This can be rewritten to the following:
public async Task<(string, SomeObject)> DoSomethingWrapperAsync(string thing)
=> (thing, await DoSomethingAsync(thing));
public async Task<(string, SomeObject)> DoSomethingElseWrapperAsync(string thing)
=> (thing, await DoSomethingElseAsync(thing));
public async Task<IEnumerable<(string, SomeObject)>>
DoManyThingsAsync(IEnumerable<string> someListOfStrings, bool condition)
{
var tasks = new List<Task<(string, SomeObject)>>();
foreach (var thing in someListOfStrings)
{
if (condition)
{
tasks.Add(DoSomethingWrapperAsync(thing));
}
else
{
tasks.Add(DoSomethingElseWrapperAsync(thing));
}
}
// this does compile
var results = await Task.WhenAll(tasks);
return results;
}
The problem is that I need an extra wrapper method for every possible discrete async function I'm going to call, which feels unnecessary and wasteful and is a lot of code (because there will be MANY of these methods). Is there a simpler way of achieving what I need?
I looked into implementing the awaitable/awaiter pattern, but can't see how I could get it to work with Task.WhenAll which requires a collection of Task or Task<TResult>, since the guidance seems to be "don't extend those classes".

You can either do the zipping as you go:
public async Task<IEnumerable<(string, SomeObject)>>
DoManyThingsAsync(IEnumerable<string> someListOfStrings, bool condition)
{
var tasks = someListOfStrings
.Select(async item =>
condition ?
(item, await DoSomethingAsync(item)) :
(item, await DoSomethingElseAsync(item)))
.ToList();
return await Task.WhenAll(tasks);
}
Or, you can keep the input as a separate collection and zip it later:
public async Task<IEnumerable<(string, SomeObject)>>
DoManyThingsAsync(IEnumerable<string> someListOfStrings, bool condition)
{
// Reify input so we don't enumerate twice.
var input = someListOfStrings.ToList();
var tasks = input
.Select(item =>
condition ?
DoSomethingAsync(item) :
DoSomethingElseAsync(item))
.ToList();
var taskResults = await Task.WhenAll(tasks);
return input.Zip(taskResults, (item, taskResult) => ((item, taskResult)));
}

From what I can gather you are using condition to determine which method (of the exact same signature) is being called. Why not pass a callback that is called for each item instead of doing the logic inside the foreach loop?
public async Task<SomeObject> DoSomethingAsync(string thing)
{
// ...
}
public async Task<SomeObject> DoSomethingElseAsync(string thing)
{
// ...
}
public async Task<IEnumerable<(string, SomeObject)>> DoManyThingsAsync(IEnumerable<string> someListOfStrings, bool condition)
{
Func<string, Task<SomeObject>> callback = condition ? DoSomethingAsync : DoSomethingElseAsync;
var results = await Task.WhenAll(
someListOfStrings.Select(async thing => (thing, await callback(thing)))
);
return results;
}
Furthermore, you can extract this as an extension method.
public static class AsyncExtensions
{
public static async Task<IEnumerable<(T, TResult)>> WhenAllAsync(this IEnumerable<T> collection, Func<T, Task<TResult>> selector)
{
var results = await Task.WhenAll(
collection.Select(async item => (item, await selector(item)))
);
return results;
}
}
public async Task MyMethodAsync()
{
// ...
var results = await myListOfStrings.WhenAllAsync(condition ? DoSomethingAsync : DoSomethingElseAsync);
// ...
}
Or do the mapping in the callback.
public static class AsyncExtensions
{
public static Task<IEnumerable<TResult>> WhenAllAsync(this IEnumerable<T> collection, Func<T, Task<TResult>> selector)
=> Task.WhenAll(collection.Select(selector)));
}
public async Task MyMethodAsync()
{
// ...
var results = await myListOfStrings.WhenAllAsync(async thing => (thing, await (condition ? DoSomethingAsync(thing) : DoSomethingElseAsync(thing))));
// ...
}

Map async result with automapper

We are createing a Web.Api application of a angularjs application. The Web.Api returns a json result.
Step one was getting the data:
public List<DataItem>> GetData()
{
return Mapper.Map<List<DataItem>>(dataRepository.GetData());
}
That worked like a charm. Then we made the data repo async and we changed to code to work with it.
public List<DataItem>> GetData()
{
return Mapper.Map<List<DataItem>>(dataRepository.GetDataAsync().Result);
}
Stil no problems. Now we wanted to make my Web.Api full async awayt so we changed it to:
public async Task<List<DataItem>> GetData()
{
return await Mapper.Map<Task<List<DataItem>>>(dataRepository.GetDataAsync());
}
At this moment Automapper gets confused. First we had the following mapper rule:
Mapper.CreateMap();
This worked until the web api method became full async. The exception said it was missing a map from
Task<ICollection<Data>> to Task<ICollection<DataItem>>.
The mapper was changed to
Mapper.CreateMap<Task<List<Data>>, Task<List<DataItem>>>();
When validating the configuration it complains that Result cannot be mapped. How should we configure the mappings?

You need to move the async data fetch out of the Map call:
var data = await dataRepository.GetDataAsync();
return Mapper.Map<List<DataItem>>(data);
Alternatively, you can use AutoMapper LINQ projections:
var data = await dbContext.Data.ProjectTo<DataItem>().ToListAsync();
I don't know if your repository exposes IQueryable directly (we don't use repositories). Our apps use the second version pretty much exclusively these days.

Jose's solution proved quite useful to me. I did, however, re-target the extension method to extend IMapper which allowed me to remove the singleton Mapper reference.
public static class MapperExtensions
{
public static Task<TResult> MapAsync<TSource, TResult>(this IMapper mapper, Task<TSource> task)
{
if (task == null)
{
throw new ArgumentNullException(nameof(task));
}
var tcs = new TaskCompletionSource<TResult>();
task
.ContinueWith(t => tcs.TrySetCanceled(), TaskContinuationOptions.OnlyOnCanceled);
task
.ContinueWith
(
t =>
{
tcs.TrySetResult(mapper.Map<TSource, TResult>(t.Result));
},
TaskContinuationOptions.OnlyOnRanToCompletion
);
task
.ContinueWith
(
t => tcs.TrySetException(t.Exception),
TaskContinuationOptions.OnlyOnFaulted
);
return tcs.Task;
}
}

You can also add a couple of Task Extension Methods that will do this for you:
public static Task<TReturn> Convert<T, TReturn>(this Task<T> task)
{
if (task == null)
throw new ArgumentNullException(nameof(task));
var tcs = new TaskCompletionSource<TReturn>();
task.ContinueWith(t => tcs.TrySetCanceled(), TaskContinuationOptions.OnlyOnCanceled);
task.ContinueWith(t =>
{
tcs.TrySetResult(Mapper.Map<T, TReturn>(t.Result));
}, TaskContinuationOptions.OnlyOnRanToCompletion);
task.ContinueWith(t => tcs.TrySetException(t.Exception), TaskContinuationOptions.OnlyOnFaulted);
return tcs.Task;
}
public static Task<List<TReturn>> ConvertEach<T, TReturn>(this Task<List<T>> task)
{
if (task == null)
throw new ArgumentNullException(nameof(task));
var tcs = new TaskCompletionSource<List<TReturn>>();
task.ContinueWith(t => tcs.TrySetCanceled(), TaskContinuationOptions.OnlyOnCanceled);
task.ContinueWith(t =>
{
tcs.TrySetResult(t.Result.Select(Mapper.Map<T, TReturn>).ToList());
}, TaskContinuationOptions.OnlyOnRanToCompletion);
task.ContinueWith(t => tcs.TrySetException(t.Exception), TaskContinuationOptions.OnlyOnFaulted);
return tcs.Task;
}

I recently encountered this issue as well. I wanted to keep the async part of the method and therefore, I ended up with a solution that would wrap the await operation inside an async task.
public async Task<ActionResult<ProjectDTO>> GetProject(long id)
{
return await Task<ProjectDTO>.Run(
async () => {
var project = await _context.MDb.FindAsync(id);
return _mapper.Map<ProjectDTO>(project);
}
);
}
I would love to hear your thoughts/comments.

await not blocking until Task finishes

I am trying to block RequestHandler.ParseAll() with await ConsumerTask;, but when i set a breakpoint there, i ALWAYS get the "Done..." output first... and then Parse2() fails with a NullReferenceException. (thats my guess: "the GC starts cleaning up because _handler got out of scope")
Anyway, I can't figure out why that happens.
class MainClass
{
public async void DoWork()
{
RequestHandler _handler = new RequestHandler();
string[] mUrls;
/* fill mUrls here with values */
await Task.Run(() => _handler.ParseSpecific(mUrls));
Console.WriteLine("Done...");
}
}
static class Parser
{
public static async Task<IEnumerable<string>> QueryWebPage(string url) { /*Query the url*/ }
public static async Task Parse1(Query query)
{
Parallel.ForEach(/*Process data here*/);
}
public static async Task Parse2(Query query)
{
foreach(string line in query.WebPage)
/* Here i get a NullReference exception because query.WebPage == null */
}
}
sealed class RequestHandler
{
private BlockingCollection<Query> Queue;
private Task ConsumerTask = Task.Run(() => /* call consume() for each elem in the queue*/);
private async void Consume(Query obj)
{
await (obj.BoolField ? Parser.Parse1(obj) : Parser.Parse2(obj));
}
public async void ParseSpecific(string[] urls)
{
foreach(string v in urls)
Queue.Add(new Query(await QueryWebPage(v), BoolField: false));
Queue.CompleteAdding();
await ConsumerTask;
await ParseAll(true);
}
private async Task ParseAll(bool onlySome)
{
ReInit();
Parallel.ForEach(mCollection, v => Queue.Add(new Query(url, BoolField:false)));
Queue.CompleteAdding();
await ConsumerTask;
/* Process stuff further */
}
}
struct Query
{
public readonly string[] WebPage;
public readonly bool BoolField;
public Query(uint e, IEnumerable<string> page, bool b) : this()
{
Webpage = page.ToArray();
BoolField = b;
}
}

CodesInChaos has spotted the problem in comments. It stems from having async methods returning void, which you should almost never do - it means you've got no way to track them.
Instead, if your async methods don't have any actual value to return, you should just make them return Task.
What's happening is that ParseSpecific is only running synchronously until the first await QueryWebPage(v) that doesn't complete immediately. It's then returning... so the task started here:
await Task.Run(() => _handler.ParseSpecific(mUrls));
... completes immediately, and "Done" gets printed.
Once you've made all your async methods return Task, you can await them. You also won't need Task.Run at all. So you'd have:
public async void DoWork()
{
RequestHandler _handler = new RequestHandler();
string[] mUrls;
await _handler.ParseSpecific(mUrls);
Console.WriteLine("Done...");
}
...
public async TaskParseSpecific(string[] urls)
{
foreach(string v in urls)
{
// Refactored for readability, although I'm not sure it really
// makes sense now that it's clearer! Are you sure this is what
// you want?
var page = await QueryWebPage(v);
Queue.Add(new Query(page, false);
}
Queue.CompleteAdding();
await ConsumerTask;
await ParseAll(true);
}
Your Reinit method also needs changing, as currently the ConsumerTask will basically complete almost immediately, as Consume will return immediately as it's another async method returning void.
To be honest, what you've got looks very complex, without a proper understanding of async/await. I would read up more on async/await and then probably start from scratch. I strongly suspect you can make this much, much simpler. You might also want to read up on TPL Dataflow which is designed to make producer/consumer scenarios simpler.

c# Can a "task method" also be an "async" method?

I'm trying to get the hand of the new async CTP stuff and I'm probably confusing myself here..
I can have this "task method", with no problem:
public static Task<String> LongTaskAAsync() {
return Task.Run(() => {
return("AAA");
});
}
But what if I need the task to execute another task, can I mark it as "async" and use "await"? I tried this:
public async static Task<String> LongTaskAAsync() {
await Task.Delay(2000);
return Task.Run(() => {
return("AAA");
});
}
But then mysteriously get this compiler error: Since this is an async method, the return expression must be of type 'string' rather than Task<string>
What am I missing here?

You may want to read my async/await intro post.
Return values from async methods are wrapped in a Task<TResult>. Likewise, await unwraps those return values:
public static async Task<String> LongTaskAAsync() {
await Task.Delay(2000);
return await Task.Run(() => {
return("AAA");
});
}
The reasoning behind this is described in my Async "Why Do the Keywords Work That Way" Unofficial FAQ.
P.S. You can also use Task.FromResult for simple tests like this.
Edit: If you want to create and return the Task object itself, then the method should not be async. One somewhat common pattern is to have a public non-async method that calls the async portion only if necessary.
For example, some kind of asynchronous cache - if the object is in the cache, then return it immediately; otherwise, asynchronously create it, add it to the cache, and return it (this is example code - not thread-safe):
public static Task<MyClass> GetAsync(int key)
{
if (cache.Contains(key))
return Task.FromResult(cache[key]);
return CreateAndAddAsync(key);
}
private static async Task<MyClass> CreateAndAddAsync(int key)
{
var result = await CreateAsync(key);
cache.Add(key, result);
return result;
}

Can a “task method” also be an “async” method?
Yes it can be, by simply changing the method signature to public async static Task<Task<String>> LongTaskAAsync() since that is, what it will return.
If you use the async keyword, the runtime will wrap the type you return into a task, to enable asynchronousness. Say if you return a string, the runtime will wrap that into a Task<string>. int will go Task<int> and Task<string> will go Task<Task<string>>. See this console app to clearify:
public class Program
{
public static void Main(string[] args)
{
// start the main procedure asynchron
Task.Run(() => DoIt()).Wait();
}
// for async support since the static main method can't be async
public static async void DoIt()
{
Program p = new Program();
// use the methods
string s = await p.GetString();
int i = await p.GetInt();
Task<string> tsk = await p.GetTaskOfString();
// just to prove the task works:
// C# 5
string resultFromReturnedTask = await tsk;
// C# 4
string resultFromReturnedTask2 = tsk.Result;
}
public async Task<string> GetString()
{
return "string";
}
public async Task<int> GetInt()
{
return 6;
}
public async Task<Task<string>> GetTaskOfString()
{
return Task.Run(() => "string");
}
}