We Keep Coding

Retrieving what did work when Task.WhenAll throws exception

Consider
string[] pages;
Task [] asyncOps =
(from url in urls select DownloadStringAsync(url)).ToArray();
try
{
pages = await Task.WhenAll(asyncOps);
...
}
catch(Exception exc)
{
foreach(Task<string> faulted in asyncOps.Where(t => t.IsFaulted))
{
… // work with faulted and faulted.Exception
}
}
from https://msdn.microsoft.com/en-us/library/hh873173%28v=vs.110%29.aspx. How can I retrieve the pages that DID work?
Or better, how can I keep going and compute the rest of the pages?

Instead of doing all the downloads and then handling each success/error separately, I think it's much cleaner if you define a separate "download and handle error" operation:
Task [] asyncOps =
(from url in urls select DownloadStringWithErrorCheckingAsync(url)).ToArray();
string[] pages = await Task.WhenAll(asyncOps);
var successfulPages = pages.Where(x => x != null);
...
private static Task<string> DownloadStringWithErrorCheckingAsync(string url)
{
try
{
return await DownloadStringAsync(url);
}
catch(Exception exc)
{
... // work with exc
return null;
}
}

I've same problem. I need to start several task, wait for each to terminate and then, process all task Status/Exception/Result.
I can't use Stephen's solution because final processings are not independent from each other. It's some kind of: if task1 is Ok, I will try to take result of task2 and if not, I will take task3 result. I need each response to infer my behavior.
Task[] toWait = new Task[]{...};
await Task.WhenAll(toWait).ContinueWith((t) => {t?.Exception?.Handle((exc)=>true);}, ct);
I don't await result of WhenAll but result of ContinueWith whose action only silently handle exception.
It's not very elegant but we can hide it with a method like this one:
public static Task WhenAllNoThrow(this Task[] toWait, CancellationToken token)
{
return TaskEx.WhenAll(toWait).ContinueWith((t) => { t?.Exception?.Handle((exc) => true); }, token);
}
EDIT: Added null-conditional Operators ?.

Its not clear what "DownloadStringAsync" is, I assumed it was WebClient.DownloadStringAsync
This function does not return a task, you need to subscribe to a the complete event to capture the result.
It doesnt appear that is what you wanted to do so I changed your code to do a parallel for loop instead.
Dictionary<string, string> pages = new Dictionary<string, string>();
Dictionary<string, string> errors = new Dictionary<string, string>();
string[] urls = new string[] { "http://www.google.com", "http://www.bbc.co.uk" };
Parallel.ForEach<string>(urls, (url) =>
{
var webClient = new System.Net.WebClient();
try
{
pages[url] = webClient.DownloadString(new Uri(url));
}
catch(Exception ex)
{
errors[url] = ex.Message;
}
});
// The successful
foreach(var kvp in pages)
{
Console.WriteLine(kvp.Key);
//Console.WriteLine(kvp.Value);
}
// The failures
foreach (var kvp in errors)
{
Console.WriteLine(kvp.Key);
//Console.WriteLine(kvp.Value);
}

await Task.WhenAll(tasks) Exception Handling, log all exceptions from the tasks

I am trying to figure out how to report all exceptions thrown by a list of tasks from the code below.
The basic idea of this code snippet is: The user sends a request to the handler, the handler creates the messages tasks and send them to a class that send them to the external system. I included the methods involved below.
I must have something off because I was debugging the exception handler and the tasks Exception are always null because it seems their status is Waiting for Activiation unless I stay in the breakpoint long enough.
// Handle the user request
public async void Handle(WriteScanToSys settings)
{
_successfulScanIds = new List<int>();
// create the messages as tasks
var tasks = _factory.CreateMessage(settings).Select(msg => SendScans(msg));
try
{
// wait for all of them to complete
await Task.WhenAll(tasks); // I had ConfigureAwait(false) here, but took it off
}
catch (Exception)
{
foreach (var task in tasks.Where(t => t.Exception != null))
{
// ELMAH
var errorLog = ErrorLog.GetDefault(null);
errorLog.Log(new Error(task.Exception));
}
}
// save to repository
}
// the task to perform
private async Task<IDictionary<string, object>> SendScans(IDictionary<string, object> data)
{
object sysMsg = null;
var response = await _msgCenter.SendMessage(data);
response.TryGetValue("SystemMessage", out sysMsg);
_successfulScanIds.Add(Convert.ToInt32(data["Id"]));
return response;
}
// the communication with the external system (The message center class)
private async Task<IDictionary<string, object>> SendMessage(IDictionary<string, object> msg)
{
var response = new Dictionary<string, object>();
var response = await _client.sendAsync(
new BodyOfRequest(
// Compose Object
));
if (response.ScreenMessage != "SUCCESSFUL")
throw new CustomException("The transaction for job " + job + " failed with msg: " + body.ScreenMessage);
response.Add("SystemMessage", body.ScreenMessage);
return response;
}

You've fallen foul of lazy evaluation - the result of Select will create a new set of tasks each time you iterate over it. You can fix this just by calling ToList():
var tasks = _factory.CreateMessage(settings)
.Select(msg => SendScans(msg))
.ToList();
That way the set of tasks that you're awaiting will be the same set of tasks checked with your foreach loop.

Instead of iterating over all tasks, you can get the Exceptions (if any) from the Task.WhenAll-Task:
var taskResult = Task.WhenAll(tasks);
try
{
await taskResult;
}
catch (Exception e)
{
if (taskResult.IsCanceled)
{
// Cancellation is most likely due to a shared cancellation token. Handle as needed, possibly check if ((TaskCanceledException)e).CancellationToken == token etc.
}
else if (taskResult.IsFaulted)
{
// use taskResult.Exception which is an AggregateException - which you can iterate over (it's a tree! .Flatten() might help)
// caught exception is only the first observed exception
}
else
{
// Well, this should not really happen because it would mean: Exception thrown, not faulted nor cancelled but completed
}
}

Gracefully handeling exceptions when dealing with WhenAll

I'm playing with a piece of code I wrote a while back. That piece of code deals with making a few requests in an async manner.
var client = new HttpClient();
var searchPromises = searchTerms
.Select(GetSearchUrl)
.Select(client.GetStringAsync);
var searchPages = await Task.WhenAll(searchPromises);
What happens is I create a new HttpClient. Using some search terch terms I compose search engine urls. Then I use those urls as inputs to get tasks representing the async requests for a page with the results. And last, I await those responses using Task.WhenAll to group them together.
The problem is if just one of those requests gets a 404, a 500 or anything like that my code throws an AggregateException.
Is there a way of specifying what should happen in the case of an error in one of those threads, so that I get a result from everything else?
I've looked at ContinueWith, but it doesn't seem to fit the bill, that is, it doesn't know how to deal with all the errors, just the aggregate one.

What happens is I create a new HttpClient. Using some search terch terms I compose search engine urls. Then I use those urls as inputs to get tasks representing the async requests for a page with the results. And last, I await those responses using Task.WhenAll to group them together.
Is there a way of specifying what should happen in the case of an error in one of those threads, so that I get a result from everything else?
IMO, the easiest solution is to change how you think about the problem. Right now, you're thinking "perform a download on each url" and then "what for them all to complete and handle errors on a per-item basis". Just change your operation ("download") to include anything you want to do per-item. In other words, what you want to do is "perform a download on each url and handle errors" and then "wait for them all to complete":
var client = new HttpClient();
var searchPromises = searchTerms
.Select(GetSearchUrl)
.Select(url => DownloadAsync(client, url));
var searchPages = await Task.WhenAll(searchPromises);
var successfulSearchPages = searchPages.Where(x => x != null);
...
private static async Task<string> DownloadAsync(HttpClient client, string url)
{
try
{
return await client.GetStringAsync(url);
}
catch (HttpRequestException ex)
{
// TODO: Perform appropriate error handling
return null;
}
}

Task.WhenAll will return a task that is completed when all the tasks passed as argument are completed.
If any of the tasks passed as argument ends in a Faulted state (an exception was thrown), the returned task will also end in a Faulted state and its Exception property will contain the aggregation of all exceptions thrown by the tasks passed as argument.
Because the code generated by the compiler picks the first exceptin on the list, only the excpetion thrown by the first exception that throws (not the first exception thrwing) will be rethrown.
But the tasks passed as argument still exist and can still be queried for result.
This code snippet shows this working:
var tasks = new Task[] {
((Func<Task>)(async () =>
{
await Task.Delay(10);
await Task.Delay(10);
await Task.Delay(10);
throw new Exception("First");
}))(),
((Func<Task>)(async () =>
{
await Task.Delay(10);
throw new Exception("Second");
}))(),
((Func<Task>)(async () =>
{
await Task.Delay(10);
}))()
};
var allTasks = Task.WhenAll(tasks);
try
{
await allTasks;
}
catch (Exception ex)
{
Console.WriteLine("Overall failed: {0}", ex.Message);
}
for(var i = 0; i < tasks.Length; i++)
{
try
{
await tasks[i];
Console.WriteLine("Taks {0} succeeded!", i);
}
catch (Exception ex)
{
Console.WriteLine("Taks {0} failed!", i);
}
}
/*
Overall failed: First
Taks 0 failed!
Taks 1 failed!
Taks 2 succeeded!
*/

You can create your own version of Task.WhenAll that returns just the results disregarding any exception using Task.WhenAny:
public static async Task<IEnumerable<TResult>> WhenAllSwallowExceptions<TResult>(IEnumerable<Task<TResult>> tasks)
{
var tasklist = tasks.ToList();
var results = new List<TResult>();
while (tasklist.Any())
{
var completedTask = await Task.WhenAny(tasklist);
try
{
results.Add(await completedTask);
}
catch (Exception e)
{
// handle
}
tasklist.Remove(completedTask);
}
return results;
}
Usage:
var searchPages = await WhenAllSwallowExceptions(searchPromises);
This waits for tasks one at a time (with Task.WhenAny) and aggregates all the results (if there are any).

I've found a way to do this, after many iterations. Tasks are starting to look like things that you need a library to abstract.
Anyway, here's the code:
var client = new HttpClient();
var exceptions = new ConcurrentBag<Exception>();
var searchPromises = searchTerms
.Select(GetSearchUrl)
.Select(client.GetStringAsync)
.Select(t=>t.Catch(e=>exceptions.Add(e)));
var searchPages = (await Task.WhenAll(searchPromises))
.Where(r => r != null);
And the implementation for Catch:
public static Task<TResult> Catch<TResult>(this Task<TResult> self, Action<Exception> exceptionHandlerTask)
{
return self.ContinueWith(s =>
{
if (!s.IsFaulted)
{
return s.Result;
}
exceptionHandlerTask(s.Exception);
return default(TResult);
},
CancellationToken.None,
TaskContinuationOptions.ExecuteSynchronously |
TaskContinuationOptions.DenyChildAttach,
TaskScheduler.Default);
}
What happens now is that it gives you a way to append a failure state function to the Task<T> promise. This allows me to still have chainability. It is a shame that c# doesn't have robust support for functional pattern matching to make this easier.
Edit: added minimal code for error logging.
Edit: separated the code for logging errors to be more generic/reusable.
Edit: separated the code for saving the errors from the Catch function.

Good pattern for exception handling when using async calls

I want to consume an Web API and I see many people recommending System.Net.Http.HttpClient.
That's fine... but I have only VS-2010, so I cannot use async/await just yet. Instead, I guess I could use Task<TResult> in combination to ContinueWith. So I tried this piece of code:
var client = new HttpClient();
client.DefaultRequestHeaders.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/json"));
client.GetStringAsync(STR_URL_SERVER_API_USERS).ContinueWith(task =>
{
var usersResultString = task.Result;
lbUsers.DataSource = JsonConvert.DeserializeObject<List<string>>(usersResultString);
});
My first observation was to realize that it doesn't generate any error if URL is not available, but maybe there will be more errors like this...
So I am trying to find a way to handle exceptions for such async calls (particularly for HttpClient). I noticed that "Task" has IsFaulted property and an AggregateException which maybe could be used, but I am not sure yet how.
Another observation was that GetStringAsync returns Task<string>, but GetAsync returns Task<HttpResponseMessage>. The latter could be maybe more useful, since it presents a StatusCode.
Could you share a pattern on how to use the async calls correctly and handle exceptions in a good way? Basic explanation would be appreciated as well.

I would not use a separate ContinueWith continuation for successful and faulted scenarios. I'd rather handle both cases in a single place, using try/catch:
task.ContinueWith(t =>
{
try
{
// this would re-throw an exception from task, if any
var result = t.Result;
// process result
lbUsers.DataSource = JsonConvert.DeserializeObject<List<string>>(result);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
lbUsers.Clear();
lbUsers.Items.Add("Error loading users!");
}
},
CancellationToken.None,
TaskContinuationOptions.None,
TaskScheduler.FromCurrentSynchronizationContext()
);
If t is a non-generic Task (rather than a Task<TResult>), you can do t.GetAwaiter().GetResult() to re-throw the original exception inside the ContinueWith lambda; t.Wait() would work too. Be prepared to handle AggregatedException, you can get to the inner exception with something like this:
catch (Exception ex)
{
while (ex is AggregatedException && ex.InnerException != null)
ex = ex.InnerException;
MessageBox.Show(ex.Message);
}
If you're dealing with a series of ContinueWith, usually you don't have to handle exceptions inside each ContinueWith. Do it once for the outermost resulting task, e.g.:
void GetThreePagesV1()
{
var httpClient = new HttpClient();
var finalTask = httpClient.GetStringAsync("http://example.com")
.ContinueWith((task1) =>
{
var page1 = task1.Result;
return httpClient.GetStringAsync("http://example.net")
.ContinueWith((task2) =>
{
var page2 = task2.Result;
return httpClient.GetStringAsync("http://example.org")
.ContinueWith((task3) =>
{
var page3 = task3.Result;
return page1 + page2 + page3;
}, TaskContinuationOptions.ExecuteSynchronously);
}, TaskContinuationOptions.ExecuteSynchronously).Unwrap();
}, TaskContinuationOptions.ExecuteSynchronously).Unwrap()
.ContinueWith((resultTask) =>
{
httpClient.Dispose();
string result = resultTask.Result;
try
{
MessageBox.Show(result);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
},
CancellationToken.None,
TaskContinuationOptions.None,
TaskScheduler.FromCurrentSynchronizationContext());
}
Any exceptions thrown inside inner tasks will propagate to the outermost ContinueWith lambda as you're accessing the results of the inner tasks (taskN.Result).
This code is functional, but it's also ugly and non-readable. JavaScript developers call it The Callback Pyramid of Doom. They have Promises to deal with it. C# developers have async/await, which you're unfortunately not able to use because of the VS2010 restrain.
IMO, the closest thing to the JavaScript Promises in TPL is Stephen Toub's Then pattern. And the closest thing to async/await in C# 4.0 is his Iterate pattern from the same blog post, which uses the C# yield feature.
Using the Iterate pattern, the above code could be rewritten in a more readable way. Note that inside GetThreePagesHelper you can use all the familiar synchronous code statements like using, for, while, try/catch etc. It is however important to understand the asynchronous code flow of this pattern:
void GetThreePagesV2()
{
Iterate(GetThreePagesHelper()).ContinueWith((iteratorTask) =>
{
try
{
var lastTask = (Task<string>)iteratorTask.Result;
var result = lastTask.Result;
MessageBox.Show(result);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
throw;
}
},
CancellationToken.None,
TaskContinuationOptions.None,
TaskScheduler.FromCurrentSynchronizationContext());
}
IEnumerable<Task> GetThreePagesHelper()
{
// now you can use "foreach", "using" etc
using (var httpClient = new HttpClient())
{
var task1 = httpClient.GetStringAsync("http://example.com");
yield return task1;
var page1 = task1.Result;
var task2 = httpClient.GetStringAsync("http://example.net");
yield return task2;
var page2 = task2.Result;
var task3 = httpClient.GetStringAsync("http://example.org");
yield return task3;
var page3 = task3.Result;
yield return Task.Delay(1000);
var resultTcs = new TaskCompletionSource<string>();
resultTcs.SetResult(page1 + page1 + page3);
yield return resultTcs.Task;
}
}
/// <summary>
/// A slightly modified version of Iterate from
/// http://blogs.msdn.com/b/pfxteam/archive/2010/11/21/10094564.aspx
/// </summary>
public static Task<Task> Iterate(IEnumerable<Task> asyncIterator)
{
if (asyncIterator == null)
throw new ArgumentNullException("asyncIterator");
var enumerator = asyncIterator.GetEnumerator();
if (enumerator == null)
throw new InvalidOperationException("asyncIterator.GetEnumerator");
var tcs = new TaskCompletionSource<Task>();
Action<Task> nextStep = null;
nextStep = (previousTask) =>
{
if (previousTask != null && previousTask.Exception != null)
tcs.SetException(previousTask.Exception);
if (enumerator.MoveNext())
{
enumerator.Current.ContinueWith(nextStep,
TaskContinuationOptions.ExecuteSynchronously);
}
else
{
tcs.SetResult(previousTask);
}
};
nextStep(null);
return tcs.Task;
}

Task crash while trying to search folder without permission

This is my function that get IEnumerable<string> source and search all the files inside this path:
public void search()
{
Task.Factory.StartNew(() =>
{
try
{
Parallel.ForEach(_source,
new ParallelOptions
{
MaxDegreeOfParallelism = 5 //limit number of parallel threads here
},
file =>
{
FileChecker fileChecker = new FileChecker();
string result = fileChecker.check(file);
if (result != null)
OnFileAddEvent(result);
});
}
catch (Exception)
{ }
}).ContinueWith
(t =>
{
OnFinishSearchEvent();
}
, TaskScheduler.FromCurrentSynchronizationContext() //to ContinueWith (update UI) from UI thread
);
}
public void search2()
{
Task.Factory.StartNew(() =>
{
var filtered = _source.AsParallel()
.WithDegreeOfParallelism(5)
.Where(file =>
{
try
{
FileChecker fileChecker = new FileChecker();
string result = fileChecker.check(file);
if (result != null)
OnFileAddEvent(result);
return true;
}
catch (Exception)
{
return false;
}
});
return filtered.ToList();
}).ContinueWith
(t =>
{
OnFinishSearchEvent();
}
, TaskScheduler.FromCurrentSynchronizationContext() //to ContinueWith (update UI) from UI thread
);
}

It looks like you have a fundamental misunderstanding of Multithreading and Exception Handling.
Exception handling in .net typically starts with a managed exception being thrown. This then causes the .net VM to then walk up the call stack until it finds an appropriate try catch block.
In this case the Task.Factory.StartNew(Action), is a wrapper to push onto the thread pool work described in the Action delegate. The top of the call stack would in this case be the Action delegate described in...
file =>
{
// check my file before adding to my Listbox
}
So when an exception bubbles up...nothing is "up" the call stack to catch it.
The solution then is either to, as others have described, add a root level try catch in the delegate you pass into the Task.Factory.StartNew(Action) method.
More generally one would normally add a OnError continuation on the Task returned by the Task.Factory.StartNew(Action)...however I would also add that I would be worried that this entire method is highly flawed, as none of the worker threads should be able to add to the ListBox. The ListBox should ONLY ever be accessed by the STA thread that constructed it.
Ultimately I would change the entire method to the following...
public void search()
{
Task.Factory.StartNew(() =>
{
var filtered = source.AsParallel()
.WithDegreeOfParallelism(5)
.Where(file =>
{
try
{
//Some filter function...
}
catch(Exception)
{
return false;
}
});
return filtered.ToList();
}).ContinueWith
(t =>
{
foreach(var result in t.Result)
{
MyListBox.Add(result);
}
OnFinishSearchEvent();
}
, TaskScheduler.FromCurrentSynchronizationContext() //to ContinueWith (update UI) from UI thread
);
}

You must catch what is inside of the Task:
public void search()
{
try
{
Task.Factory.StartNew(() =>
{
try{
Parallel.ForEach(source,
new ParallelOptions
{
MaxDegreeOfParallelism = 5 //limit number of parallel threads here
},
file =>
{
// check my file before adding to my Listbox
});
}).ContinueWith
(t =>
{
OnFinishSearchEvent();
}
, TaskScheduler.FromCurrentSynchronizationContext() //to ContinueWith (update UI) from UI thread
);
catch(Exception ex){
}
}
catch (Exception ex)
{
}
}