How to cancel IAsyncEnumerable when exception occurs? - c#

I am using IAsyncEnumerable to fetch a lot of data - like this:
static async IAsyncEnumerable<int> GetSequence([EnumeratorCancellation] CancellationToken ct = default)
{
for (int i = 1; i <= 10; i++)
{
var data = await DoSomethingAsync() // get data from web request
yield return data;
}
}
Now here is my question: if DoSomethingAsync throws an exception (maybe a network issue), I would like to cancel the remaining requests, is it possible?
I am trying below code to handle but still need help:
CancellationTokenSource source = new CancellationTokenSource();
CancellationToken token = source.Token;
try
{
IAsyncEnumerable<int> dataCol = GetSequence(token);
await foreach (var d in dataCol)
{
// do ...
}
}
catch (Exception)
{
source.Cancel(true);
throw;
}
Appreciate if someone has best practices

From the original language proposal
await foreach (var d in dataCol)
{
// do ...
}
is equivalent to;
{
E e = dataCol.GetAsyncEnumerator();
try {
while (await e.MoveNextAsync()) {
var d = e.Current;
// do ...
}
}
finally {
... // Dispose e
}
}
Calling MoveNextAsync causes your GetSequence method to execute up to the next yield return, or until an unhandled exception is thrown. Execution of that method will not resume until the next call to MoveNextAsync. GetSequence and await foreach do not run in parallel, they take turns executing more code within the same async task.
if DoSomethingAsync throws an exception
then await e.MoveNextAsync() will re-throw the exception. Then enumerator will be disposed. Then the exception would flow up the stack to the next exception handler.
You don't need to mess around with cancelation tokens to prevent GetSequence from resuming execution. In fact breaking out of the await foreach loop would also abort execution, as if the yield return never resumed. Causing any finally{} blocks in your GetSequence method to execute, if you had any.

Related

Calling AWS RDS CreateDBSnapshotAsync Asynchronously "Set It And Forget It"

In an AWS Lambda function, I would like to be able to call a component to create a RDS DB Snapshot. There is an async method on the client named CreateDBSnapshotAsync. But, because this is AWS Lambda, I only have 5 minutes to complete the task. So, if I await it, the AWS Lambda function will timeout. And, apparently when it times out, the call is cancelled and then the snapshot is not completed.
Is there some way I can make the call in a COMPLETELY asynchronously way so that once I invoke it, it will complete no matter if my Lambda function times out or not?
In other words, I don't care about the result, I just want to invoke the process and move on, a "set it and forget it" mentality.
My call (without the await, obviously) is as below
using (var rdsClient = new AmazonRDSClient())
{
Task<CreateDBSnapshotResponse> response = rdsClient.CreateDBSnapshotAsync(new CreateDBSnapshotRequest($"MySnapShot", instanceId));
}
As requested, here's the full method:
public async Task<CloudFormationResponse> MigrateDatabase(CloudFormationRequest request, ILambdaContext context)
{
LambdaLogger.Log($"{nameof(MigrateDatabase)} invoked: " + JsonConvert.SerializeObject(request));
if (request.RequestType != "Delete")
{
try
{
var migrations = this.Context.Database.GetPendingMigrations().OrderBy(b=>b).ToList();
for (int i = 0; i < migrations.Count(); i++)
{
string thisMigration = migrations [i];
this.ApplyMigrationInternal(thisMigration);
}
this.TakeSnapshotAsync(context,migrations.Last());
return await CloudFormationResponse.CompleteCloudFormationResponse(null, request, context);
}
catch (Exception e)
{
LambdaLogger.Log(e.ToString());
if (e.InnerException != null) LambdaLogger.Log(e.InnerException.ToString());
return await CloudFormationResponse.CompleteCloudFormationResponse(e, request, context);
}
}
return await CloudFormationResponse.CompleteCloudFormationResponse(null, request, context);
}
internal void TakeSnapshotAsync(ILambdaContext context, string migration)
{
var instanceId = this.GetEnvironmentVariable(nameof(DBInstance));
using (var rdsClient = new AmazonRDSClient())
{
Task<CreateDBSnapshotResponse> response = rdsClient.CreateDBSnapshotAsync(new CreateDBSnapshotRequest($"{instanceId}{migration.Replace('_','-')}", instanceId));
while (context.RemainingTime > TimeSpan.FromSeconds(15))
{
Thread.Sleep(15000);
}
}
}
First refactor that sub function to use proper async syntax along with the use of Task.WhenAny.
internal async Task TakeSnapshotAsync(ILambdaContext context, string migration) {
var instanceId = this.GetEnvironmentVariable(nameof(DBInstance));
//don't wrap in using block or it will be disposed before you are done with it.
var rdsClient = new AmazonRDSClient();
var request = new CreateDBSnapshotRequest($"{instanceId}{migration.Replace('_','-')}", instanceId);
//don't await this long running task
Task<CreateDBSnapshotResponse> response = rdsClient.CreateDBSnapshotAsync(request);
Task delay = Task.Run(async () => {
while (context.RemainingTime > TimeSpan.FromSeconds(15)) {
await Task.Delay(15000); //Don't mix Thread.Sleep. use Task.Delay and await it.
}
}
// The call returns as soon as the first operation completes,
// even if the others are still running.
await Task.WhenAny(response, delay);
}
So if the RemainingTime runs out, it will break out of the call even if the snap shot task is still running so that the request does not time out.
Now you should be able to await the snapshot while there is still time available in the context
public async Task<CloudFormationResponse> MigrateDatabase(CloudFormationRequest request, ILambdaContext context) {
LambdaLogger.Log($"{nameof(MigrateDatabase)} invoked: " + JsonConvert.SerializeObject(request));
if (request.RequestType != "Delete") {
try {
var migrations = this.Context.Database.GetPendingMigrations().OrderBy(b=>b).ToList();
for (int i = 0; i < migrations.Count(); i++) {
string thisMigration = migrations [i];
this.ApplyMigrationInternal(thisMigration);
}
await this.TakeSnapshotAsync(context, migrations.Last());
return await CloudFormationResponse.CompleteCloudFormationResponse(null, request, context);
} catch (Exception e) {
LambdaLogger.Log(e.ToString());
if (e.InnerException != null) LambdaLogger.Log(e.InnerException.ToString());
return await CloudFormationResponse.CompleteCloudFormationResponse(e, request, context);
}
}
return await CloudFormationResponse.CompleteCloudFormationResponse(null, request, context);
}
This should also allow for any exceptions thrown by the RDS client to be caught by the currently executing thread. Which should help with troubleshooting any exception messages.
Some interesting information from documentation.
Using Async in C# Functions with AWS Lambda
If you know your Lambda function will require a long-running process, such as uploading large files to Amazon S3 or reading a large stream of records from DynamoDB, you can take advantage of the async/await pattern. When you use this signature, Lambda executes the function synchronously and waits for the function to return a response or for execution to time out.
From docs about timeouts
Function Settings
...
Timeout – The amount of time that Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds.
If getting a HTTP timeout then shorten the delay but leave the long running task. You still use the Task.WhenAny to give the long running task an opportunity to finish first even if that is not the expectation.
internal async Task TakeSnapshotAsync(ILambdaContext context, string migration) {
var instanceId = this.GetEnvironmentVariable(nameof(DBInstance));
//don't wrap in using block or it will be disposed before you are done with it.
var rdsClient = new AmazonRDSClient();
var request = new CreateDBSnapshotRequest($"{instanceId}{migration.Replace('_','-')}", instanceId);
//don't await this long running task
Task<CreateDBSnapshotResponse> response = rdsClient.CreateDBSnapshotAsync(request);
Task delay = Task.Delay(TimeSpan.FromSeconds(2.5));
// The call returns as soon as the first operation completes,
// even if the others are still running.
await Task.WhenAny(response, delay);
}

Exception from PLinq AsParallel async crashes the app

My question is about capturing exceptions in ForAll method under Plinq
I was trying to run tasks concurently with setting max number of threads.
Using
enumerable
.AsParallel()
.WithDegreeOfParallelism(100)
.ForAll(async item => await AsyncTask())
It works, but if AsyncTask throws exception the app crashes.
I have done the following test:
try
{
IEnumerable<string> enumerable = new List<string> { "st", "st" };
enumerable.AsParallel()
.ForAll(async f =>
{
try
{
throw new Exception(); // Or await AsyncTask that throws this
}
catch (Exception e)
{
e.ToString(); **// This Exception is captured**
throw e;
}
});
}
catch (Exception e) **// THIS IS NOT CAPTURED AND THE APP CRASHES**
{
e.ToString();
}
And I would like to understand the reasons for this
And other options to implement
enumerable.AsParallel().ForAll() executes the given action for each element of your enumeration in parallel. Since your given action is async by itself, ForAll() does not wait until all actions completed. In this case the executed code leaves the try..catch block before your AsyncTask() method throws the exception. This may lead to an unhandled exception, which crashes your app.
It does not matter, that you try to await the AsyncTask(), because ForAll() gets a plain Action and does not await the result of your AsyncTask().
A possible solution could be to start your AsyncTasks for each element without AsParallel().ForEach() and await the results later inside your try..catch.
When storing the
Task or Task<T>
result in a list you can check if any task was throwing an exception using the task.Exception property.
You can do something like this:
private async Task DoSomethingAsync()
{
try
{
IEnumerable<string> enumerable = new List<string> { "st", "st" };
// start all tasks and store them in an array
var tasks = enumerable.Select(TaskAsync).ToArray();
// do something more without waiting until all tasks above completed
// ...
// await all tasks
var completionTask = Task.WhenAll(tasks);
await completionTask;
// handle task exception if any exists
if (completionTask.Status == TaskStatus.Faulted)
{
foreach (var task in tasks)
{
if (task.Exception != null)
{
// throw an exception or handle the exception, e.g. log the exceptions to file / database
}
}
}
}
catch (Exception e)
{
// handle your exception, e.g. write a log to file / database
}
}
private Task TaskAsync(string item)
{
// Task.Delay() is just a placeholder
// do some async stuff here, e.g. access web services or a database
return Task.Delay(10000);
}

await Task.WhenAll(tasks) Exception Handling, log all exceptions from the tasks

I am trying to figure out how to report all exceptions thrown by a list of tasks from the code below.
The basic idea of this code snippet is: The user sends a request to the handler, the handler creates the messages tasks and send them to a class that send them to the external system. I included the methods involved below.
I must have something off because I was debugging the exception handler and the tasks Exception are always null because it seems their status is Waiting for Activiation unless I stay in the breakpoint long enough.
// Handle the user request
public async void Handle(WriteScanToSys settings)
{
_successfulScanIds = new List<int>();
// create the messages as tasks
var tasks = _factory.CreateMessage(settings).Select(msg => SendScans(msg));
try
{
// wait for all of them to complete
await Task.WhenAll(tasks); // I had ConfigureAwait(false) here, but took it off
}
catch (Exception)
{
foreach (var task in tasks.Where(t => t.Exception != null))
{
// ELMAH
var errorLog = ErrorLog.GetDefault(null);
errorLog.Log(new Error(task.Exception));
}
}
// save to repository
}
// the task to perform
private async Task<IDictionary<string, object>> SendScans(IDictionary<string, object> data)
{
object sysMsg = null;
var response = await _msgCenter.SendMessage(data);
response.TryGetValue("SystemMessage", out sysMsg);
_successfulScanIds.Add(Convert.ToInt32(data["Id"]));
return response;
}
// the communication with the external system (The message center class)
private async Task<IDictionary<string, object>> SendMessage(IDictionary<string, object> msg)
{
var response = new Dictionary<string, object>();
var response = await _client.sendAsync(
new BodyOfRequest(
// Compose Object
));
if (response.ScreenMessage != "SUCCESSFUL")
throw new CustomException("The transaction for job " + job + " failed with msg: " + body.ScreenMessage);
response.Add("SystemMessage", body.ScreenMessage);
return response;
}
You've fallen foul of lazy evaluation - the result of Select will create a new set of tasks each time you iterate over it. You can fix this just by calling ToList():
var tasks = _factory.CreateMessage(settings)
.Select(msg => SendScans(msg))
.ToList();
That way the set of tasks that you're awaiting will be the same set of tasks checked with your foreach loop.
Instead of iterating over all tasks, you can get the Exceptions (if any) from the Task.WhenAll-Task:
var taskResult = Task.WhenAll(tasks);
try
{
await taskResult;
}
catch (Exception e)
{
if (taskResult.IsCanceled)
{
// Cancellation is most likely due to a shared cancellation token. Handle as needed, possibly check if ((TaskCanceledException)e).CancellationToken == token etc.
}
else if (taskResult.IsFaulted)
{
// use taskResult.Exception which is an AggregateException - which you can iterate over (it's a tree! .Flatten() might help)
// caught exception is only the first observed exception
}
else
{
// Well, this should not really happen because it would mean: Exception thrown, not faulted nor cancelled but completed
}
}

Gracefully handeling exceptions when dealing with WhenAll

I'm playing with a piece of code I wrote a while back. That piece of code deals with making a few requests in an async manner.
var client = new HttpClient();
var searchPromises = searchTerms
.Select(GetSearchUrl)
.Select(client.GetStringAsync);
var searchPages = await Task.WhenAll(searchPromises);
What happens is I create a new HttpClient. Using some search terch terms I compose search engine urls. Then I use those urls as inputs to get tasks representing the async requests for a page with the results. And last, I await those responses using Task.WhenAll to group them together.
The problem is if just one of those requests gets a 404, a 500 or anything like that my code throws an AggregateException.
Is there a way of specifying what should happen in the case of an error in one of those threads, so that I get a result from everything else?
I've looked at ContinueWith, but it doesn't seem to fit the bill, that is, it doesn't know how to deal with all the errors, just the aggregate one.
What happens is I create a new HttpClient. Using some search terch terms I compose search engine urls. Then I use those urls as inputs to get tasks representing the async requests for a page with the results. And last, I await those responses using Task.WhenAll to group them together.
Is there a way of specifying what should happen in the case of an error in one of those threads, so that I get a result from everything else?
IMO, the easiest solution is to change how you think about the problem. Right now, you're thinking "perform a download on each url" and then "what for them all to complete and handle errors on a per-item basis". Just change your operation ("download") to include anything you want to do per-item. In other words, what you want to do is "perform a download on each url and handle errors" and then "wait for them all to complete":
var client = new HttpClient();
var searchPromises = searchTerms
.Select(GetSearchUrl)
.Select(url => DownloadAsync(client, url));
var searchPages = await Task.WhenAll(searchPromises);
var successfulSearchPages = searchPages.Where(x => x != null);
...
private static async Task<string> DownloadAsync(HttpClient client, string url)
{
try
{
return await client.GetStringAsync(url);
}
catch (HttpRequestException ex)
{
// TODO: Perform appropriate error handling
return null;
}
}
Task.WhenAll will return a task that is completed when all the tasks passed as argument are completed.
If any of the tasks passed as argument ends in a Faulted state (an exception was thrown), the returned task will also end in a Faulted state and its Exception property will contain the aggregation of all exceptions thrown by the tasks passed as argument.
Because the code generated by the compiler picks the first exceptin on the list, only the excpetion thrown by the first exception that throws (not the first exception thrwing) will be rethrown.
But the tasks passed as argument still exist and can still be queried for result.
This code snippet shows this working:
var tasks = new Task[] {
((Func<Task>)(async () =>
{
await Task.Delay(10);
await Task.Delay(10);
await Task.Delay(10);
throw new Exception("First");
}))(),
((Func<Task>)(async () =>
{
await Task.Delay(10);
throw new Exception("Second");
}))(),
((Func<Task>)(async () =>
{
await Task.Delay(10);
}))()
};
var allTasks = Task.WhenAll(tasks);
try
{
await allTasks;
}
catch (Exception ex)
{
Console.WriteLine("Overall failed: {0}", ex.Message);
}
for(var i = 0; i < tasks.Length; i++)
{
try
{
await tasks[i];
Console.WriteLine("Taks {0} succeeded!", i);
}
catch (Exception ex)
{
Console.WriteLine("Taks {0} failed!", i);
}
}
/*
Overall failed: First
Taks 0 failed!
Taks 1 failed!
Taks 2 succeeded!
*/
You can create your own version of Task.WhenAll that returns just the results disregarding any exception using Task.WhenAny:
public static async Task<IEnumerable<TResult>> WhenAllSwallowExceptions<TResult>(IEnumerable<Task<TResult>> tasks)
{
var tasklist = tasks.ToList();
var results = new List<TResult>();
while (tasklist.Any())
{
var completedTask = await Task.WhenAny(tasklist);
try
{
results.Add(await completedTask);
}
catch (Exception e)
{
// handle
}
tasklist.Remove(completedTask);
}
return results;
}
Usage:
var searchPages = await WhenAllSwallowExceptions(searchPromises);
This waits for tasks one at a time (with Task.WhenAny) and aggregates all the results (if there are any).
I've found a way to do this, after many iterations. Tasks are starting to look like things that you need a library to abstract.
Anyway, here's the code:
var client = new HttpClient();
var exceptions = new ConcurrentBag<Exception>();
var searchPromises = searchTerms
.Select(GetSearchUrl)
.Select(client.GetStringAsync)
.Select(t=>t.Catch(e=>exceptions.Add(e)));
var searchPages = (await Task.WhenAll(searchPromises))
.Where(r => r != null);
And the implementation for Catch:
public static Task<TResult> Catch<TResult>(this Task<TResult> self, Action<Exception> exceptionHandlerTask)
{
return self.ContinueWith(s =>
{
if (!s.IsFaulted)
{
return s.Result;
}
exceptionHandlerTask(s.Exception);
return default(TResult);
},
CancellationToken.None,
TaskContinuationOptions.ExecuteSynchronously |
TaskContinuationOptions.DenyChildAttach,
TaskScheduler.Default);
}
What happens now is that it gives you a way to append a failure state function to the Task<T> promise. This allows me to still have chainability. It is a shame that c# doesn't have robust support for functional pattern matching to make this easier.
Edit: added minimal code for error logging.
Edit: separated the code for logging errors to be more generic/reusable.
Edit: separated the code for saving the errors from the Catch function.

Strange execution jump when using async/await and System.Threading.Tasks.Parallel

I have the following method:
public async Task ExecuteAsync()
{
Task<IEnumerable<Comment>> gettingComments = RetrieveComments();
Dictionary<string, ReviewManager> reviewers = ConfigurationFacade.Repositories.ToDictionary(name => name, name => new ReviewManager(name));
IEnumerable<Comment> comments = await gettingComments;
Parallel.ForEach(reviewers, (reviewer) => {
Dictionary<Comment, RevisionResult> reviews = reviewer.Value.Review(comments);
int amountModerated = ModerateComments(reviews.Where(r => r.Value.IsInsult), "hide");
});
}
My ModerateComments method looks like the following:
private Task<int> ModerateComments(IEnumerable<Comment> comments, string operation)
{
return Task.Factory.StartNew(() =>
{
int moderationCount = 0;
Parallel.ForEach(comments, async (comment) =>
{
bool moderated = await ModerateComment(comment, operation); //Problem here
if(moderated)
moderationCount++;
}
return moderationCount;
};
}
And finally:
private async Task<bool> ModerateComment(Comment comment, string operation, string authenticationToken = null)
{
if(comment == null) return false;
if(String.IsNullOrWhiteSpace(authenticationToken))
authenticationToken = CreateUserToken(TimeSpan.FromMinutes(1));
string moderationEndpoint = ConfigurationFacade.ModerationEndpoint;
using(HttpRequestMessage request = new HttpRequestMessage())
{
request.Method = HttpMethod.Post;
request.RequestUri = new Uri(moderationEndpoint);
using(HttpResponseMessage response = await _httpClient.SendAsync(request)) //Problem here
{
if(!response.IsSuccessStatusCode)
{
if(response.StatusCode == HttpStatusCode.Unauthorized)
return await ModerateComment(comment, operation, null); //Retry operation with a new access token
else if(response.StatusCode == HttpStatusCode.GatewayTimeout)
return await ModerateComment(comment, operation, authenticationToken); //Retry operation
return false;
}
}
}
return true;
}
I'm having a strange problem at runtime. All the above code is working fine except when it reaches the line:
using(HttpResponseMessage response = await _httpClient.SendAsync(request)) {
//...
}
When I debug my application, this instruction is executed but just after that, it does not throw any exception, nor return anything, it just finish executing and I am derived to the next statement on the Parallel.ForEach loop.
It is really hard to explain so I'll post some images:
All good so far, I reach the following line of code:
The execution keeps going well and I reach the call to the Moderation API
Even if I press F10 (Next statement) in the debugger, the execution flow jumps to the next loop in the Parallel.ForEach loop.
As you can see I have breakpoints in the try-catch just i ncase any exception is thrown, but the breakpoint is never activated, neither is activated the breakpoint in if(moderacion) commentCount++.
So what happens here? Where did my execution flow went? It just dissapears after sending the POST request to the API.
After continuing the execution, all the elements in the enumerable do the same jump, and therefore, my commentCount variable ends up being equal to 0
You don't need Parallel.ForEach or Task.Factory.StartNew to do IO bound work:
private async Task<int> ModerateCommentsAsync(IEnumerable<Comment> comments, string operation)
{
var commentTasks = comments.Select(comment => ModerateCommentAsync(comment, operation));
await Task.WhenAll(commentTasks);
return commentTasks.Count(x => x.Result);
}
Common practice is to add the Async postfix to an async method.
Excellent description for a common problem. Parallel.ForEach does not support async lambdas. async methods return once they hit the first await that would need to block. This happens when you issue the HTTP request.
Use one of the common patterns for a parallel async foreach loop.

Categories