I am trying to figure out how to report all exceptions thrown by a list of tasks from the code below.
The basic idea of this code snippet is: The user sends a request to the handler, the handler creates the messages tasks and send them to a class that send them to the external system. I included the methods involved below.
I must have something off because I was debugging the exception handler and the tasks Exception are always null because it seems their status is Waiting for Activiation unless I stay in the breakpoint long enough.
// Handle the user request
public async void Handle(WriteScanToSys settings)
{
_successfulScanIds = new List<int>();
// create the messages as tasks
var tasks = _factory.CreateMessage(settings).Select(msg => SendScans(msg));
try
{
// wait for all of them to complete
await Task.WhenAll(tasks); // I had ConfigureAwait(false) here, but took it off
}
catch (Exception)
{
foreach (var task in tasks.Where(t => t.Exception != null))
{
// ELMAH
var errorLog = ErrorLog.GetDefault(null);
errorLog.Log(new Error(task.Exception));
}
}
// save to repository
}
// the task to perform
private async Task<IDictionary<string, object>> SendScans(IDictionary<string, object> data)
{
object sysMsg = null;
var response = await _msgCenter.SendMessage(data);
response.TryGetValue("SystemMessage", out sysMsg);
_successfulScanIds.Add(Convert.ToInt32(data["Id"]));
return response;
}
// the communication with the external system (The message center class)
private async Task<IDictionary<string, object>> SendMessage(IDictionary<string, object> msg)
{
var response = new Dictionary<string, object>();
var response = await _client.sendAsync(
new BodyOfRequest(
// Compose Object
));
if (response.ScreenMessage != "SUCCESSFUL")
throw new CustomException("The transaction for job " + job + " failed with msg: " + body.ScreenMessage);
response.Add("SystemMessage", body.ScreenMessage);
return response;
}
You've fallen foul of lazy evaluation - the result of Select will create a new set of tasks each time you iterate over it. You can fix this just by calling ToList():
var tasks = _factory.CreateMessage(settings)
.Select(msg => SendScans(msg))
.ToList();
That way the set of tasks that you're awaiting will be the same set of tasks checked with your foreach loop.
Instead of iterating over all tasks, you can get the Exceptions (if any) from the Task.WhenAll-Task:
var taskResult = Task.WhenAll(tasks);
try
{
await taskResult;
}
catch (Exception e)
{
if (taskResult.IsCanceled)
{
// Cancellation is most likely due to a shared cancellation token. Handle as needed, possibly check if ((TaskCanceledException)e).CancellationToken == token etc.
}
else if (taskResult.IsFaulted)
{
// use taskResult.Exception which is an AggregateException - which you can iterate over (it's a tree! .Flatten() might help)
// caught exception is only the first observed exception
}
else
{
// Well, this should not really happen because it would mean: Exception thrown, not faulted nor cancelled but completed
}
}
Related
I am using IAsyncEnumerable to fetch a lot of data - like this:
static async IAsyncEnumerable<int> GetSequence([EnumeratorCancellation] CancellationToken ct = default)
{
for (int i = 1; i <= 10; i++)
{
var data = await DoSomethingAsync() // get data from web request
yield return data;
}
}
Now here is my question: if DoSomethingAsync throws an exception (maybe a network issue), I would like to cancel the remaining requests, is it possible?
I am trying below code to handle but still need help:
CancellationTokenSource source = new CancellationTokenSource();
CancellationToken token = source.Token;
try
{
IAsyncEnumerable<int> dataCol = GetSequence(token);
await foreach (var d in dataCol)
{
// do ...
}
}
catch (Exception)
{
source.Cancel(true);
throw;
}
Appreciate if someone has best practices
From the original language proposal
await foreach (var d in dataCol)
{
// do ...
}
is equivalent to;
{
E e = dataCol.GetAsyncEnumerator();
try {
while (await e.MoveNextAsync()) {
var d = e.Current;
// do ...
}
}
finally {
... // Dispose e
}
}
Calling MoveNextAsync causes your GetSequence method to execute up to the next yield return, or until an unhandled exception is thrown. Execution of that method will not resume until the next call to MoveNextAsync. GetSequence and await foreach do not run in parallel, they take turns executing more code within the same async task.
if DoSomethingAsync throws an exception
then await e.MoveNextAsync() will re-throw the exception. Then enumerator will be disposed. Then the exception would flow up the stack to the next exception handler.
You don't need to mess around with cancelation tokens to prevent GetSequence from resuming execution. In fact breaking out of the await foreach loop would also abort execution, as if the yield return never resumed. Causing any finally{} blocks in your GetSequence method to execute, if you had any.
My question is about capturing exceptions in ForAll method under Plinq
I was trying to run tasks concurently with setting max number of threads.
Using
enumerable
.AsParallel()
.WithDegreeOfParallelism(100)
.ForAll(async item => await AsyncTask())
It works, but if AsyncTask throws exception the app crashes.
I have done the following test:
try
{
IEnumerable<string> enumerable = new List<string> { "st", "st" };
enumerable.AsParallel()
.ForAll(async f =>
{
try
{
throw new Exception(); // Or await AsyncTask that throws this
}
catch (Exception e)
{
e.ToString(); **// This Exception is captured**
throw e;
}
});
}
catch (Exception e) **// THIS IS NOT CAPTURED AND THE APP CRASHES**
{
e.ToString();
}
And I would like to understand the reasons for this
And other options to implement
enumerable.AsParallel().ForAll() executes the given action for each element of your enumeration in parallel. Since your given action is async by itself, ForAll() does not wait until all actions completed. In this case the executed code leaves the try..catch block before your AsyncTask() method throws the exception. This may lead to an unhandled exception, which crashes your app.
It does not matter, that you try to await the AsyncTask(), because ForAll() gets a plain Action and does not await the result of your AsyncTask().
A possible solution could be to start your AsyncTasks for each element without AsParallel().ForEach() and await the results later inside your try..catch.
When storing the
Task or Task<T>
result in a list you can check if any task was throwing an exception using the task.Exception property.
You can do something like this:
private async Task DoSomethingAsync()
{
try
{
IEnumerable<string> enumerable = new List<string> { "st", "st" };
// start all tasks and store them in an array
var tasks = enumerable.Select(TaskAsync).ToArray();
// do something more without waiting until all tasks above completed
// ...
// await all tasks
var completionTask = Task.WhenAll(tasks);
await completionTask;
// handle task exception if any exists
if (completionTask.Status == TaskStatus.Faulted)
{
foreach (var task in tasks)
{
if (task.Exception != null)
{
// throw an exception or handle the exception, e.g. log the exceptions to file / database
}
}
}
}
catch (Exception e)
{
// handle your exception, e.g. write a log to file / database
}
}
private Task TaskAsync(string item)
{
// Task.Delay() is just a placeholder
// do some async stuff here, e.g. access web services or a database
return Task.Delay(10000);
}
I have implemented a service name ExamClient which have two operations one is Ping which return a basic string which mean service is available and one is FindStudy which search in DB it may take a long to be proceeded.
In the other side I have several endpoints of ExamClient I wand to run FindStudy per end point by task so in a Dispatcher I have something like this:
public FindStudies_DTO_OUT FindStudies(FindStudies_DTO_IN findStudies_DTO_IN)
{
List<Study_C> ret = new List<Study_C>();
List<Task> tasks = new List<Task>();
foreach (var sp in Cluster)
{
string serviceAddress = sp.GetLibraryAddress(ServiceLibrary_C.PCM) + "/Exam.svc";
var task = Task.Run(() =>
{
ExamClient examClient = new ExamClient(serviceAddress.GetBinding(), new EndpointAddress(serviceAddress), Token);
var ping = Task.Run(() =>
{
examClient.Ping();
});
if (!ping.Wait(examClient.Endpoint.Binding.OpenTimeout))
{
Logging.Log(LoggingMode.Warning, "Timeout on FindStudies for:{0}, address:{1}", sp.Name, serviceAddress);
return new List<Study_C>(); // if return null then need to manage it on ret.AddRange(t.Result);
}
return (examClient.FindStudies(findStudies_DTO_IN).Studies.Select(x =>
{
x.StudyInstanceUID = string.Format("{0}|{1}", sp.Name, x.StudyInstanceUID);
x.InstitutionName = sp.Name;
return x;
}));
});
task.ContinueWith(t =>
{
lock (ret)
{
ret.AddRange(t.Result);
}
}, TaskContinuationOptions.OnlyOnRanToCompletion);
task.ContinueWith(t =>
{
Logging.Log(LoggingMode.Error, "FindStudies failed for :{0}, address:{1}, EXP:{2}", sp.Name, serviceAddress, t.Exception.ToString());
}, TaskContinuationOptions.OnlyOnFaulted);
tasks.Add(task);
}
try
{
Task.WaitAll(tasks.ToArray());
}
catch (AggregateException aggEx)
{
foreach (Exception exp in aggEx.InnerExceptions)
{
Logging.Log(LoggingMode.Error, "Error while FindStudies EXP:{0}", exp.ToString());
}
}
return new FindStudies_DTO_OUT(ret.Sort(findStudies_DTO_IN.SortColumnName, findStudies_DTO_IN.SortOrderBy));
}
First I have to run Ping per end point to know connection is established
after that FindStudy.
if there are three end pints in Cluster six task be run in parallel mode, 3 for Ping and 3 for FindStudy.
I think something is wrong with my code to handle exception nice...
So what is the best way to implement this scenario ?
thanks in advance.
Let me throw my answer to simplify and remove unnecessary code block. And bit of explanation along the code.
public FindStudies_DTO_OUT FindStudies(FindStudies_DTO_IN findStudies_DTO_IN)
{
// Thread-safe collection
var ret = new ConcurrentBag<Study_C>()
// Loop cluster list and process each item in parallel and wait all process to finish. This handle the parallism better than task run
Parallel.Foreach(Cluster, (sp) =>
{
var serviceAddress = sp.GetLibraryAddress(ServiceLibrary_C.PCM) + "/Exam.svc";
ExamClient examClient = new ExamClient(serviceAddress.GetBinding(), new EndpointAddress(serviceAddress), Token);
try
{
examClient.Ping();
// declare result variable type outside try catch to be visible below
var result = examClient.FindStudies(findStudies_DTO_IN);
}
catch(TimeoutException timeoutEx)
{
// abort examclient to dispose channel properly
Logging.Log(LoggingMode.Warning, "Timeout on FindStudies for:{0}, address:{1}", sp.Name, serviceAddress);
}
catch(FaultException fault)
{
Logging.Log(LoggingMode.Error, "FindStudies failed for :{0}, address:{1}, EXP:{2}", sp.Name, serviceAddress, fault.Exception.ToString());
}
catch(Exception ex)
{
// anything else
}
// add exception type as needed for proper logging
// use inverted if to reduce nested condition
if( result == null )
return null;
var study_c = result.Studies.Select(x =>
{
x.StudyInstanceUID = string.Format("{0}|{1}", sp.Name, x.StudyInstanceUID);
x.InstitutionName = sp.Name;
return x;
}));
// Thread-safe collection
ret.AddRange(study_c);
});
// for sorting i guess concurrentBag has orderby; if not working convert to list
return new FindStudies_DTO_OUT(ret.Sort(findStudies_DTO_IN.SortColumnName, findStudies_DTO_IN.SortOrderBy));
}
Note : Code haven't tested but the gist is there. Also I feels like task.run inside task.run is bad idea can't remember which article I read it (probably from Stephen Cleary not sure).
I have a function (below) that I retrieve data from an API. If I set a breakpoint at the line that deserializes it, then I can see that it is populated with data which is great.
When I continue on, it goes into the second function (below) and it throws an error. The error says next to it Not yet computed, and therefore throwing an exception.
When I do it with a small list it works just fine (I presume its cos it's a small set of data).
How is this possible when I'm using ContinueWith (waiting for the task to complete)?
public static async Task<Data> GetAllCardsInSet(string setName)
{
setName = WebUtility.UrlEncode(setName);
var correctUri = Path.Combine(ApiConstants.YugiohGetAllCardsInSet, setName);
Console.WriteLine();
using (var httpClient = new HttpClient())
{
var response =
await httpClient.GetAsync(correctUri);
var result = await response.Content.ReadAsStringAsync();
var cardData = JsonConvert.DeserializeObject<CardSetCards>(result);
for (int i = 0; i < cardData.Data.Cards.Count; i++)
{
cardData.Data.Cards[i] = FormatWords(cardData.Data.Cards[i]);
}
return cardData.Data;
}
}
private void GetYugiohCardsAndNavigate(string name)
{
var cardSetData = YugiohRequester.GetAllCardsInSet(_selectedCardSet.Name).ContinueWith((result) =>
{
//var cards = await YugiohRequester.GetAllCardsInSet(_selectedCardSet.Name);
try
{
this.mainPage.NavigateToYugiohCardListPage(result.Result);
}
catch (Exception e)
{
HelperFunctions.ShowToastNotification("Trading Card App", "Sorry, we could not fetch this set");
}
});
}
Your GetAllCardsInSet method no need to change.
But using of this method can be refactored.
Method GetAllCardsInSet return Task and you not observed the completion of the this Task.
You need to check is Task completes succesfully, easiest approach to use await keyword. Awaiting task will unwrapp returned value or throw exception if task completed with exception.
For using async/await in the GetYugiohCardsAndNavigate change method signature to aynchronous and returning Task
private async Task GetYugiohCardsAndNavigate(string name)
{
try
{
var cardSetData = await YugiohRequester.GetAllCardsInSet(_selectedCardSet.Name);
this.mainPage.NavigateToYugiohCardListPage(cardSetData);
}
catch (Exception e)
{
HelperFunctions.ShowToastNotification("Trading Card App",
"Sorry, we could not fetch this set");
}
}
you called an async method in a sync method without Wait. It should have been done like:
YugiohRequester.GetAllCardsInSet(_selectedCardSet.Name).ContinueWith((result) =>
{
//var cards = await YugiohRequester.GetAllCardsInSet(_selectedCardSet.Name);
try
{
this.mainPage.NavigateToYugiohCardListPage(result.Result);
}
catch (Exception e)
{
HelperFunctions.ShowToastNotification("Trading Card App", "Sorry, we could not fetch this set");
}
}).Wait();
I'm playing with a piece of code I wrote a while back. That piece of code deals with making a few requests in an async manner.
var client = new HttpClient();
var searchPromises = searchTerms
.Select(GetSearchUrl)
.Select(client.GetStringAsync);
var searchPages = await Task.WhenAll(searchPromises);
What happens is I create a new HttpClient. Using some search terch terms I compose search engine urls. Then I use those urls as inputs to get tasks representing the async requests for a page with the results. And last, I await those responses using Task.WhenAll to group them together.
The problem is if just one of those requests gets a 404, a 500 or anything like that my code throws an AggregateException.
Is there a way of specifying what should happen in the case of an error in one of those threads, so that I get a result from everything else?
I've looked at ContinueWith, but it doesn't seem to fit the bill, that is, it doesn't know how to deal with all the errors, just the aggregate one.
What happens is I create a new HttpClient. Using some search terch terms I compose search engine urls. Then I use those urls as inputs to get tasks representing the async requests for a page with the results. And last, I await those responses using Task.WhenAll to group them together.
Is there a way of specifying what should happen in the case of an error in one of those threads, so that I get a result from everything else?
IMO, the easiest solution is to change how you think about the problem. Right now, you're thinking "perform a download on each url" and then "what for them all to complete and handle errors on a per-item basis". Just change your operation ("download") to include anything you want to do per-item. In other words, what you want to do is "perform a download on each url and handle errors" and then "wait for them all to complete":
var client = new HttpClient();
var searchPromises = searchTerms
.Select(GetSearchUrl)
.Select(url => DownloadAsync(client, url));
var searchPages = await Task.WhenAll(searchPromises);
var successfulSearchPages = searchPages.Where(x => x != null);
...
private static async Task<string> DownloadAsync(HttpClient client, string url)
{
try
{
return await client.GetStringAsync(url);
}
catch (HttpRequestException ex)
{
// TODO: Perform appropriate error handling
return null;
}
}
Task.WhenAll will return a task that is completed when all the tasks passed as argument are completed.
If any of the tasks passed as argument ends in a Faulted state (an exception was thrown), the returned task will also end in a Faulted state and its Exception property will contain the aggregation of all exceptions thrown by the tasks passed as argument.
Because the code generated by the compiler picks the first exceptin on the list, only the excpetion thrown by the first exception that throws (not the first exception thrwing) will be rethrown.
But the tasks passed as argument still exist and can still be queried for result.
This code snippet shows this working:
var tasks = new Task[] {
((Func<Task>)(async () =>
{
await Task.Delay(10);
await Task.Delay(10);
await Task.Delay(10);
throw new Exception("First");
}))(),
((Func<Task>)(async () =>
{
await Task.Delay(10);
throw new Exception("Second");
}))(),
((Func<Task>)(async () =>
{
await Task.Delay(10);
}))()
};
var allTasks = Task.WhenAll(tasks);
try
{
await allTasks;
}
catch (Exception ex)
{
Console.WriteLine("Overall failed: {0}", ex.Message);
}
for(var i = 0; i < tasks.Length; i++)
{
try
{
await tasks[i];
Console.WriteLine("Taks {0} succeeded!", i);
}
catch (Exception ex)
{
Console.WriteLine("Taks {0} failed!", i);
}
}
/*
Overall failed: First
Taks 0 failed!
Taks 1 failed!
Taks 2 succeeded!
*/
You can create your own version of Task.WhenAll that returns just the results disregarding any exception using Task.WhenAny:
public static async Task<IEnumerable<TResult>> WhenAllSwallowExceptions<TResult>(IEnumerable<Task<TResult>> tasks)
{
var tasklist = tasks.ToList();
var results = new List<TResult>();
while (tasklist.Any())
{
var completedTask = await Task.WhenAny(tasklist);
try
{
results.Add(await completedTask);
}
catch (Exception e)
{
// handle
}
tasklist.Remove(completedTask);
}
return results;
}
Usage:
var searchPages = await WhenAllSwallowExceptions(searchPromises);
This waits for tasks one at a time (with Task.WhenAny) and aggregates all the results (if there are any).
I've found a way to do this, after many iterations. Tasks are starting to look like things that you need a library to abstract.
Anyway, here's the code:
var client = new HttpClient();
var exceptions = new ConcurrentBag<Exception>();
var searchPromises = searchTerms
.Select(GetSearchUrl)
.Select(client.GetStringAsync)
.Select(t=>t.Catch(e=>exceptions.Add(e)));
var searchPages = (await Task.WhenAll(searchPromises))
.Where(r => r != null);
And the implementation for Catch:
public static Task<TResult> Catch<TResult>(this Task<TResult> self, Action<Exception> exceptionHandlerTask)
{
return self.ContinueWith(s =>
{
if (!s.IsFaulted)
{
return s.Result;
}
exceptionHandlerTask(s.Exception);
return default(TResult);
},
CancellationToken.None,
TaskContinuationOptions.ExecuteSynchronously |
TaskContinuationOptions.DenyChildAttach,
TaskScheduler.Default);
}
What happens now is that it gives you a way to append a failure state function to the Task<T> promise. This allows me to still have chainability. It is a shame that c# doesn't have robust support for functional pattern matching to make this easier.
Edit: added minimal code for error logging.
Edit: separated the code for logging errors to be more generic/reusable.
Edit: separated the code for saving the errors from the Catch function.