Consider
string[] pages;
Task [] asyncOps =
(from url in urls select DownloadStringAsync(url)).ToArray();
try
{
pages = await Task.WhenAll(asyncOps);
...
}
catch(Exception exc)
{
foreach(Task<string> faulted in asyncOps.Where(t => t.IsFaulted))
{
… // work with faulted and faulted.Exception
}
}
from https://msdn.microsoft.com/en-us/library/hh873173%28v=vs.110%29.aspx. How can I retrieve the pages that DID work?
Or better, how can I keep going and compute the rest of the pages?
Instead of doing all the downloads and then handling each success/error separately, I think it's much cleaner if you define a separate "download and handle error" operation:
Task [] asyncOps =
(from url in urls select DownloadStringWithErrorCheckingAsync(url)).ToArray();
string[] pages = await Task.WhenAll(asyncOps);
var successfulPages = pages.Where(x => x != null);
...
private static Task<string> DownloadStringWithErrorCheckingAsync(string url)
{
try
{
return await DownloadStringAsync(url);
}
catch(Exception exc)
{
... // work with exc
return null;
}
}
I've same problem. I need to start several task, wait for each to terminate and then, process all task Status/Exception/Result.
I can't use Stephen's solution because final processings are not independent from each other. It's some kind of: if task1 is Ok, I will try to take result of task2 and if not, I will take task3 result. I need each response to infer my behavior.
Task[] toWait = new Task[]{...};
await Task.WhenAll(toWait).ContinueWith((t) => {t?.Exception?.Handle((exc)=>true);}, ct);
I don't await result of WhenAll but result of ContinueWith whose action only silently handle exception.
It's not very elegant but we can hide it with a method like this one:
public static Task WhenAllNoThrow(this Task[] toWait, CancellationToken token)
{
return TaskEx.WhenAll(toWait).ContinueWith((t) => { t?.Exception?.Handle((exc) => true); }, token);
}
EDIT: Added null-conditional Operators ?.
Its not clear what "DownloadStringAsync" is, I assumed it was WebClient.DownloadStringAsync
This function does not return a task, you need to subscribe to a the complete event to capture the result.
It doesnt appear that is what you wanted to do so I changed your code to do a parallel for loop instead.
Dictionary<string, string> pages = new Dictionary<string, string>();
Dictionary<string, string> errors = new Dictionary<string, string>();
string[] urls = new string[] { "http://www.google.com", "http://www.bbc.co.uk" };
Parallel.ForEach<string>(urls, (url) =>
{
var webClient = new System.Net.WebClient();
try
{
pages[url] = webClient.DownloadString(new Uri(url));
}
catch(Exception ex)
{
errors[url] = ex.Message;
}
});
// The successful
foreach(var kvp in pages)
{
Console.WriteLine(kvp.Key);
//Console.WriteLine(kvp.Value);
}
// The failures
foreach (var kvp in errors)
{
Console.WriteLine(kvp.Key);
//Console.WriteLine(kvp.Value);
}
Related
I am trying to optimize this code to decrease the time taken to complete the forloop. In this case, CreateNotification() takes a long time and using async await does not improve performance as each asynchronous call is being awaited. I would like to use Task.WhenAll() to optimize the code. How can I do this?
foreach (var notification in notificationsInput.Notifications)
{
try
{
var result = await CreateNotification(notification);
notification.Result = result;
}
catch (Exception exception)
{
notification.Result = null;
}
notifications.Add(notification);
}
You can call Select on the collection whose elements you want to process in parallel, passing an asynchronous delegate to it. This asynchronous delegate would return a Task for each element that's processed, so you could then call Task.WhenAll on all these tasks. The pattern is like so:
var tasks = collection.Select(async (x) => await ProcessAsync(x));
await Task.WhenAll(tasks);
For your example:
var tasks = notificationsInput.Notifications.Select(async (notification) =>
{
try
{
var result = await CreateNotification(notification);
notification.Result = result;
}
catch (Exception exception)
{
notification.Result = null;
}
});
await Task.WhenAll(tasks);
This assumes that CreateNotification is thread-safe.
Update
You will need to install DataFlow to use this solution
https://www.nuget.org/packages/System.Threading.Tasks.Dataflow/
Depending on what CreateNotification is and whether you want to run this in parallel.
You could use a DataFlow ActionBlock, it will give you the best of both worlds if this is IO bound or Mix IO/CPU bound operations and let you run async and in parallel
public static async Task DoWorkLoads(NotificationsInput notificationsInput)
{
var options = new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 50
};
var block = new ActionBlock<Something>(MyMethodAsync, options);
foreach (var notification in notificationsInput.Notifications)
block.Post(notification);
block.Complete();
await block.Completion;
}
...
public async Task MyMethodAsync(Notification notification)
{
var result = await CreateNotification(notification);
notification.Result = result;
}
Add pepper and salt to taste.
I think this ought to be equivalent to your code:
var notifications = new ConcurrentBag<Notification>();
var tasks = new List<Task>();
foreach (var notification in notificationsInput.Notifications)
{
var task = CreateNotification(notification)
.ContinueWith(t =>
{
if (t.Exception != null)
{
notification.Result = null;
}
else
{
notification.Result = t.Result;
}
notifications.Add(notification);
});
tasks.Add(task);
}
await Task.WhenAll(tasks);
.ContinueWith( will receive the completed/failed task from CreateNotification(, and is itself a task. We add the ContinueWith task to a list and use that in the WhenAll(.
I'm using a ConcurrentBag for notifications so that you can add from multiple threads safely. If you want to turn this into a regular list, you can call var regularListNotifications = notifications.ToList(); (assuming you have a using for LINQ).
I have implemented a service name ExamClient which have two operations one is Ping which return a basic string which mean service is available and one is FindStudy which search in DB it may take a long to be proceeded.
In the other side I have several endpoints of ExamClient I wand to run FindStudy per end point by task so in a Dispatcher I have something like this:
public FindStudies_DTO_OUT FindStudies(FindStudies_DTO_IN findStudies_DTO_IN)
{
List<Study_C> ret = new List<Study_C>();
List<Task> tasks = new List<Task>();
foreach (var sp in Cluster)
{
string serviceAddress = sp.GetLibraryAddress(ServiceLibrary_C.PCM) + "/Exam.svc";
var task = Task.Run(() =>
{
ExamClient examClient = new ExamClient(serviceAddress.GetBinding(), new EndpointAddress(serviceAddress), Token);
var ping = Task.Run(() =>
{
examClient.Ping();
});
if (!ping.Wait(examClient.Endpoint.Binding.OpenTimeout))
{
Logging.Log(LoggingMode.Warning, "Timeout on FindStudies for:{0}, address:{1}", sp.Name, serviceAddress);
return new List<Study_C>(); // if return null then need to manage it on ret.AddRange(t.Result);
}
return (examClient.FindStudies(findStudies_DTO_IN).Studies.Select(x =>
{
x.StudyInstanceUID = string.Format("{0}|{1}", sp.Name, x.StudyInstanceUID);
x.InstitutionName = sp.Name;
return x;
}));
});
task.ContinueWith(t =>
{
lock (ret)
{
ret.AddRange(t.Result);
}
}, TaskContinuationOptions.OnlyOnRanToCompletion);
task.ContinueWith(t =>
{
Logging.Log(LoggingMode.Error, "FindStudies failed for :{0}, address:{1}, EXP:{2}", sp.Name, serviceAddress, t.Exception.ToString());
}, TaskContinuationOptions.OnlyOnFaulted);
tasks.Add(task);
}
try
{
Task.WaitAll(tasks.ToArray());
}
catch (AggregateException aggEx)
{
foreach (Exception exp in aggEx.InnerExceptions)
{
Logging.Log(LoggingMode.Error, "Error while FindStudies EXP:{0}", exp.ToString());
}
}
return new FindStudies_DTO_OUT(ret.Sort(findStudies_DTO_IN.SortColumnName, findStudies_DTO_IN.SortOrderBy));
}
First I have to run Ping per end point to know connection is established
after that FindStudy.
if there are three end pints in Cluster six task be run in parallel mode, 3 for Ping and 3 for FindStudy.
I think something is wrong with my code to handle exception nice...
So what is the best way to implement this scenario ?
thanks in advance.
Let me throw my answer to simplify and remove unnecessary code block. And bit of explanation along the code.
public FindStudies_DTO_OUT FindStudies(FindStudies_DTO_IN findStudies_DTO_IN)
{
// Thread-safe collection
var ret = new ConcurrentBag<Study_C>()
// Loop cluster list and process each item in parallel and wait all process to finish. This handle the parallism better than task run
Parallel.Foreach(Cluster, (sp) =>
{
var serviceAddress = sp.GetLibraryAddress(ServiceLibrary_C.PCM) + "/Exam.svc";
ExamClient examClient = new ExamClient(serviceAddress.GetBinding(), new EndpointAddress(serviceAddress), Token);
try
{
examClient.Ping();
// declare result variable type outside try catch to be visible below
var result = examClient.FindStudies(findStudies_DTO_IN);
}
catch(TimeoutException timeoutEx)
{
// abort examclient to dispose channel properly
Logging.Log(LoggingMode.Warning, "Timeout on FindStudies for:{0}, address:{1}", sp.Name, serviceAddress);
}
catch(FaultException fault)
{
Logging.Log(LoggingMode.Error, "FindStudies failed for :{0}, address:{1}, EXP:{2}", sp.Name, serviceAddress, fault.Exception.ToString());
}
catch(Exception ex)
{
// anything else
}
// add exception type as needed for proper logging
// use inverted if to reduce nested condition
if( result == null )
return null;
var study_c = result.Studies.Select(x =>
{
x.StudyInstanceUID = string.Format("{0}|{1}", sp.Name, x.StudyInstanceUID);
x.InstitutionName = sp.Name;
return x;
}));
// Thread-safe collection
ret.AddRange(study_c);
});
// for sorting i guess concurrentBag has orderby; if not working convert to list
return new FindStudies_DTO_OUT(ret.Sort(findStudies_DTO_IN.SortColumnName, findStudies_DTO_IN.SortOrderBy));
}
Note : Code haven't tested but the gist is there. Also I feels like task.run inside task.run is bad idea can't remember which article I read it (probably from Stephen Cleary not sure).
Two questions regarding the code from below:
is this implementation an implementation of an async method?
is it the right implemention of logging an exception that might appear during deletion a file?
public static async Task DeleteFiles(StorageFolder folder, Regex mask, LoggingChannel logger)
{
var results = (from file in await folder.GetFilesAsync() where mask.IsMatch(file.Name) select file).Select(async f => await f.DeleteAsync());
await Task.WhenAll(results);
foreach (var failed in results.Where(r => r.Exception != null)) logger.LogMessage(failed.Exception.ToString(), LoggingLevel.Warning);
}
is it better (right?) way:
public static async Task DeleteFiles(StorageFolder folder, Regex mask, LoggingChannel logger)
{
foreach(var f in (await folder.GetFilesAsync()).Where( f => mask.IsMatch(f.Name)))
{
try
{
await f.DeleteAsync();
}
catch(Exception ex)
{
logger.LogMessage(ex.ToString(), LoggingLevel.Warning);
}
}
}
The second implementation will delete the files one by one since every delete will be awaited giving a synchronous experience. So the first implementation might be quicker.
Proper implementation:
public static async Task DeleteFilesAsync(StorageFolder folder, Regex mask, LoggingChannel logger)
{
var results = (from file in await folder.GetFilesAsync() where mask.IsMatch(file.Name) select file).Select(f => f.DeleteAsync());
try
{
await Task.WhenAll(results);
}
catch(Exception ex)
{
foreach (var failed in results.Where(r => r.Exception != null)) logger.LogMessage(failed.Exception.ToString(), LoggingLevel.Warning);
}
}
I think exception handling should always be clearly identified in the code, hence the try/catch block.
Mind the .Select to get the tasks, I removed the unnecessary async/await there.
or do Task.WaitAll(results) and catch the AggregateException
See also Why doesn't await on Task.WhenAll throw an AggregateException?
Another approach where logging can go in parallel. This is using rx and DataFlowBlock.
var actBlock = new ActionBlock<Task>(t => { var tsk = t.IsFaulted ? loggingTask : dummyTask; },new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = configurableValue
});
var obsr = actBlock.AsObserver();
FileDeletionTasks.ToObservable().Subscribe(t => obsr.OnNext(t), async ex => await loggingTask);
actBlock.Complete();
I'm playing with a piece of code I wrote a while back. That piece of code deals with making a few requests in an async manner.
var client = new HttpClient();
var searchPromises = searchTerms
.Select(GetSearchUrl)
.Select(client.GetStringAsync);
var searchPages = await Task.WhenAll(searchPromises);
What happens is I create a new HttpClient. Using some search terch terms I compose search engine urls. Then I use those urls as inputs to get tasks representing the async requests for a page with the results. And last, I await those responses using Task.WhenAll to group them together.
The problem is if just one of those requests gets a 404, a 500 or anything like that my code throws an AggregateException.
Is there a way of specifying what should happen in the case of an error in one of those threads, so that I get a result from everything else?
I've looked at ContinueWith, but it doesn't seem to fit the bill, that is, it doesn't know how to deal with all the errors, just the aggregate one.
What happens is I create a new HttpClient. Using some search terch terms I compose search engine urls. Then I use those urls as inputs to get tasks representing the async requests for a page with the results. And last, I await those responses using Task.WhenAll to group them together.
Is there a way of specifying what should happen in the case of an error in one of those threads, so that I get a result from everything else?
IMO, the easiest solution is to change how you think about the problem. Right now, you're thinking "perform a download on each url" and then "what for them all to complete and handle errors on a per-item basis". Just change your operation ("download") to include anything you want to do per-item. In other words, what you want to do is "perform a download on each url and handle errors" and then "wait for them all to complete":
var client = new HttpClient();
var searchPromises = searchTerms
.Select(GetSearchUrl)
.Select(url => DownloadAsync(client, url));
var searchPages = await Task.WhenAll(searchPromises);
var successfulSearchPages = searchPages.Where(x => x != null);
...
private static async Task<string> DownloadAsync(HttpClient client, string url)
{
try
{
return await client.GetStringAsync(url);
}
catch (HttpRequestException ex)
{
// TODO: Perform appropriate error handling
return null;
}
}
Task.WhenAll will return a task that is completed when all the tasks passed as argument are completed.
If any of the tasks passed as argument ends in a Faulted state (an exception was thrown), the returned task will also end in a Faulted state and its Exception property will contain the aggregation of all exceptions thrown by the tasks passed as argument.
Because the code generated by the compiler picks the first exceptin on the list, only the excpetion thrown by the first exception that throws (not the first exception thrwing) will be rethrown.
But the tasks passed as argument still exist and can still be queried for result.
This code snippet shows this working:
var tasks = new Task[] {
((Func<Task>)(async () =>
{
await Task.Delay(10);
await Task.Delay(10);
await Task.Delay(10);
throw new Exception("First");
}))(),
((Func<Task>)(async () =>
{
await Task.Delay(10);
throw new Exception("Second");
}))(),
((Func<Task>)(async () =>
{
await Task.Delay(10);
}))()
};
var allTasks = Task.WhenAll(tasks);
try
{
await allTasks;
}
catch (Exception ex)
{
Console.WriteLine("Overall failed: {0}", ex.Message);
}
for(var i = 0; i < tasks.Length; i++)
{
try
{
await tasks[i];
Console.WriteLine("Taks {0} succeeded!", i);
}
catch (Exception ex)
{
Console.WriteLine("Taks {0} failed!", i);
}
}
/*
Overall failed: First
Taks 0 failed!
Taks 1 failed!
Taks 2 succeeded!
*/
You can create your own version of Task.WhenAll that returns just the results disregarding any exception using Task.WhenAny:
public static async Task<IEnumerable<TResult>> WhenAllSwallowExceptions<TResult>(IEnumerable<Task<TResult>> tasks)
{
var tasklist = tasks.ToList();
var results = new List<TResult>();
while (tasklist.Any())
{
var completedTask = await Task.WhenAny(tasklist);
try
{
results.Add(await completedTask);
}
catch (Exception e)
{
// handle
}
tasklist.Remove(completedTask);
}
return results;
}
Usage:
var searchPages = await WhenAllSwallowExceptions(searchPromises);
This waits for tasks one at a time (with Task.WhenAny) and aggregates all the results (if there are any).
I've found a way to do this, after many iterations. Tasks are starting to look like things that you need a library to abstract.
Anyway, here's the code:
var client = new HttpClient();
var exceptions = new ConcurrentBag<Exception>();
var searchPromises = searchTerms
.Select(GetSearchUrl)
.Select(client.GetStringAsync)
.Select(t=>t.Catch(e=>exceptions.Add(e)));
var searchPages = (await Task.WhenAll(searchPromises))
.Where(r => r != null);
And the implementation for Catch:
public static Task<TResult> Catch<TResult>(this Task<TResult> self, Action<Exception> exceptionHandlerTask)
{
return self.ContinueWith(s =>
{
if (!s.IsFaulted)
{
return s.Result;
}
exceptionHandlerTask(s.Exception);
return default(TResult);
},
CancellationToken.None,
TaskContinuationOptions.ExecuteSynchronously |
TaskContinuationOptions.DenyChildAttach,
TaskScheduler.Default);
}
What happens now is that it gives you a way to append a failure state function to the Task<T> promise. This allows me to still have chainability. It is a shame that c# doesn't have robust support for functional pattern matching to make this easier.
Edit: added minimal code for error logging.
Edit: separated the code for logging errors to be more generic/reusable.
Edit: separated the code for saving the errors from the Catch function.
I want to consume an Web API and I see many people recommending System.Net.Http.HttpClient.
That's fine... but I have only VS-2010, so I cannot use async/await just yet. Instead, I guess I could use Task<TResult> in combination to ContinueWith. So I tried this piece of code:
var client = new HttpClient();
client.DefaultRequestHeaders.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/json"));
client.GetStringAsync(STR_URL_SERVER_API_USERS).ContinueWith(task =>
{
var usersResultString = task.Result;
lbUsers.DataSource = JsonConvert.DeserializeObject<List<string>>(usersResultString);
});
My first observation was to realize that it doesn't generate any error if URL is not available, but maybe there will be more errors like this...
So I am trying to find a way to handle exceptions for such async calls (particularly for HttpClient). I noticed that "Task" has IsFaulted property and an AggregateException which maybe could be used, but I am not sure yet how.
Another observation was that GetStringAsync returns Task<string>, but GetAsync returns Task<HttpResponseMessage>. The latter could be maybe more useful, since it presents a StatusCode.
Could you share a pattern on how to use the async calls correctly and handle exceptions in a good way? Basic explanation would be appreciated as well.
I would not use a separate ContinueWith continuation for successful and faulted scenarios. I'd rather handle both cases in a single place, using try/catch:
task.ContinueWith(t =>
{
try
{
// this would re-throw an exception from task, if any
var result = t.Result;
// process result
lbUsers.DataSource = JsonConvert.DeserializeObject<List<string>>(result);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
lbUsers.Clear();
lbUsers.Items.Add("Error loading users!");
}
},
CancellationToken.None,
TaskContinuationOptions.None,
TaskScheduler.FromCurrentSynchronizationContext()
);
If t is a non-generic Task (rather than a Task<TResult>), you can do t.GetAwaiter().GetResult() to re-throw the original exception inside the ContinueWith lambda; t.Wait() would work too. Be prepared to handle AggregatedException, you can get to the inner exception with something like this:
catch (Exception ex)
{
while (ex is AggregatedException && ex.InnerException != null)
ex = ex.InnerException;
MessageBox.Show(ex.Message);
}
If you're dealing with a series of ContinueWith, usually you don't have to handle exceptions inside each ContinueWith. Do it once for the outermost resulting task, e.g.:
void GetThreePagesV1()
{
var httpClient = new HttpClient();
var finalTask = httpClient.GetStringAsync("http://example.com")
.ContinueWith((task1) =>
{
var page1 = task1.Result;
return httpClient.GetStringAsync("http://example.net")
.ContinueWith((task2) =>
{
var page2 = task2.Result;
return httpClient.GetStringAsync("http://example.org")
.ContinueWith((task3) =>
{
var page3 = task3.Result;
return page1 + page2 + page3;
}, TaskContinuationOptions.ExecuteSynchronously);
}, TaskContinuationOptions.ExecuteSynchronously).Unwrap();
}, TaskContinuationOptions.ExecuteSynchronously).Unwrap()
.ContinueWith((resultTask) =>
{
httpClient.Dispose();
string result = resultTask.Result;
try
{
MessageBox.Show(result);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
},
CancellationToken.None,
TaskContinuationOptions.None,
TaskScheduler.FromCurrentSynchronizationContext());
}
Any exceptions thrown inside inner tasks will propagate to the outermost ContinueWith lambda as you're accessing the results of the inner tasks (taskN.Result).
This code is functional, but it's also ugly and non-readable. JavaScript developers call it The Callback Pyramid of Doom. They have Promises to deal with it. C# developers have async/await, which you're unfortunately not able to use because of the VS2010 restrain.
IMO, the closest thing to the JavaScript Promises in TPL is Stephen Toub's Then pattern. And the closest thing to async/await in C# 4.0 is his Iterate pattern from the same blog post, which uses the C# yield feature.
Using the Iterate pattern, the above code could be rewritten in a more readable way. Note that inside GetThreePagesHelper you can use all the familiar synchronous code statements like using, for, while, try/catch etc. It is however important to understand the asynchronous code flow of this pattern:
void GetThreePagesV2()
{
Iterate(GetThreePagesHelper()).ContinueWith((iteratorTask) =>
{
try
{
var lastTask = (Task<string>)iteratorTask.Result;
var result = lastTask.Result;
MessageBox.Show(result);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
throw;
}
},
CancellationToken.None,
TaskContinuationOptions.None,
TaskScheduler.FromCurrentSynchronizationContext());
}
IEnumerable<Task> GetThreePagesHelper()
{
// now you can use "foreach", "using" etc
using (var httpClient = new HttpClient())
{
var task1 = httpClient.GetStringAsync("http://example.com");
yield return task1;
var page1 = task1.Result;
var task2 = httpClient.GetStringAsync("http://example.net");
yield return task2;
var page2 = task2.Result;
var task3 = httpClient.GetStringAsync("http://example.org");
yield return task3;
var page3 = task3.Result;
yield return Task.Delay(1000);
var resultTcs = new TaskCompletionSource<string>();
resultTcs.SetResult(page1 + page1 + page3);
yield return resultTcs.Task;
}
}
/// <summary>
/// A slightly modified version of Iterate from
/// http://blogs.msdn.com/b/pfxteam/archive/2010/11/21/10094564.aspx
/// </summary>
public static Task<Task> Iterate(IEnumerable<Task> asyncIterator)
{
if (asyncIterator == null)
throw new ArgumentNullException("asyncIterator");
var enumerator = asyncIterator.GetEnumerator();
if (enumerator == null)
throw new InvalidOperationException("asyncIterator.GetEnumerator");
var tcs = new TaskCompletionSource<Task>();
Action<Task> nextStep = null;
nextStep = (previousTask) =>
{
if (previousTask != null && previousTask.Exception != null)
tcs.SetException(previousTask.Exception);
if (enumerator.MoveNext())
{
enumerator.Current.ContinueWith(nextStep,
TaskContinuationOptions.ExecuteSynchronously);
}
else
{
tcs.SetResult(previousTask);
}
};
nextStep(null);
return tcs.Task;
}