I am trying to understand exception handling in TPL.
The following code seems to swallow exceptions:
var processor = new ActionBlock<int>((id) => SomeAction(id), new ExecutionDataflowBlockOptions { ... });
async Task SomeAction(int merchantID)
{
//Exception producing code
...
}
And listening to TaskScheduler.UnobservedTaskException events does not receive anything either.
So, does this mean the action block does a try-catch in itself when running the actions?
Is there any official documentation of this somewhere?
Update
The exception handling behavior of DataFlow blocks is explained in Exception Handling in TPL DataFlow Networks
**Original
This code doesn't swallow exceptions. If you await the block to complete with await processor.Completion you'll get the exception. If you use a loop to pump messages to the block before calling Complete() you need a way to stop the loop too. One way to do it is to use a CancellationTokenSource and signal it in case of exception:
void SomeAction(int i,CancellationTokenSource cts)
{
try
{
...
}
catch(Exception exc)
{
//Log the error then
cts.Cancel();
//Optionally throw
}
}
The posting code doesn't have to change all that much, it only needs to check whether
var cts=new CancellationTokenSource();
var token=cts.Token;
var dopOptions=new new ExecutionDataflowBlockOptions {
MaxDegreeOfParallelism=10,
CancellationToken=token
};
var block= new ActioBlock<int>(i=>SomeAction(i,cts),dopOptions);
while(!token.IsCancellationRequested && someCondition)
{
block.Post(...);
}
block.Complete();
await block.Completion;
When the action throws, the token is signaled and the block ends. If the exception is rethrown by the action, it will be rethrown by await block.Completion as well.
If that seems convoluted, it's because that's somewhat of an edge case for blocks. DataFlow is used to create pipelines or networks of blocks.
The general case
The name Dataflow is significant.
Instead of building a program by using methods that call each other, you have processing blocks that pass messages to each other. There's no parent method to receive results and exceptions. The pipeline of blocks remains active to receive and process messages indefinitely, until some external controller tells it to stop, eg by calling Complete on the head block, or signaling the CancellationToken passed to each block.
A block shouldn't allow unhandled exceptions to occur, even if it's a standalone ActionBlock. As you saw, unless you've already called Complete() and await Completion, you won't get the exception.
When an unhandled exception occurs inside a block, the block enters the faulted state. That state propagates to all downstream blocks that are linked with the PropagateCompletion option. Upstream blocks aren't affected, which means they may keep working, storing messages in their output buffers until the process runs out of memory, or deadlocks because it receives no responses from the blocks.
Proper Failure Handling
The block should catch exceptions and decide what to do, based on the application's logic:
Log it and keep processing. That's not that different from how web application's work - an exception during a request doesn't bring down the server.
Send an error message to another block, explicitly. This works but this type of hard-coding isn't very dataflow-ish.
Use message types with some kind of error indicator. Perhaps a Success flag, perhaps an Envelope<TMessage> object that contains either a message or an error.
Gracefully cancel the entire pipeline, by signaling all blocks to cancel by signaling the CancellationTokenSource used to produce the CancellationTokens used by all blocks. That's the equivalent of throw in a common program.
#3 is the most versatile option. Downstream blocks can inspect the Envelope and ignore or propagate failed messages without processing. Essentially, failed messages bypass downstream blocks.
Another option is to use the predicate parameter in LinkTo and send failed messages to a logger block and successful messages to the next downstream block. In complex scenarios, this could be used to eg retry some operations and send the result downstream.
These concepts, and the image, come from Scott Wlaschin's Railway Oriented Programming
The TaskScheduler.UnobservedTaskException event is not a reliable/deterministic way to handle exceptions of faulted tasks, because it's delayed until the faulted task is cleaned up by the garbage collector. And this may happen long after the error occurred.
The only type of exception that is swallowed by the dataflow blocks is the OperationCanceledException (AFAIK for non-documented reasons). All other exceptions result to the block transitioning to a faulted state. A faulted block has its Completion property (which is a Task) faulted as well (processor.Completion.IsFaulted == true). You can attach a continuation to the Completion property, to receive a notification when a block fails. For example you could ensure that an exception will not pass unnoticed, by simply crashing the process:
processor.Completion.ContinueWith(t =>
{
ThreadPool.QueueUserWorkItem(_ => throw t.Exception);
}, default, TaskContinuationOptions.OnlyOnFaulted, TaskScheduler.Default);
This works because throwing an unhandled exception on the ThreadPool causes the application to terminate (after raising the AppDomain.CurrentDomain.UnhandledException event).
If your application has a GUI (WinForms/WPF etc), then you could throw the exception on the UI thread, that allows more graceful error handling:
var uiContext = SynchronizationContext.Current;
processor.Completion.ContinueWith(t =>
{
uiContext.Post(_ => throw t.Exception, null);
}, default, TaskContinuationOptions.OnlyOnFaulted, TaskScheduler.Default);
This will raise the Application.ThreadException event in WinForms.
Related
I have an IObservable set up, and I'm trying to trigger it to transition from "cold" to "hot". However, by pure dumb luck, I found that somewhere in the IObservable, an unhandled exception is throw.
I immediately freaked out because Reactive Extensions didn't let me know about this unhandled exception.
I guess the stream just terminated, instead. Now I'm worried that Rx Extensions is going to be swallowing other exceptions.
This is the code I'm currently using to trigger the IObservable to be "hot". I would expect that any unhandled exceptions that occur inside the IObservable to bubble up and be thrown here. But they are not.
var observable = Observable.Create<>(async a =>
{
...
a.OnNext();
...
a.OnCompleted();
});
observable = observable.Do(onNext: ...,
onCompleted: async () =>
{
// This throws the unhandled exception
await MethodThatThrowsExceptionAsync();
});
// I would expect any exceptions inside the IObservable to bubble up and be rethrown here.
await observable.LastOrDefaultAsync();
Am I doing something wrong? Is this expected behavior? This seems extremely error-prone if it is.
Do is for expressing side-effects on the notification, and not to actually modify the notifications themselves. Throwing errors in Do is generally not advised.
If we break this down, you want to run a task when the observable is completed, in which you may throw an exception, which bubbles down.
There's an operator which can shunt another observable once it's completed - Concat.
var observable = Observable.Create<int>(async a =>
{
await Task.Delay(1000);
a.OnNext(0);
a.OnCompleted();
});
observable = observable.Concat(Observable.FromAsync(async () =>
{
await Task.Delay(1000); //simulate work
// This throws the unhandled exception
throw new Exception("I'm from async");
return 1; //type inference
}));
// This now throws
await observable.LastOrDefaultAsync();
P.S.
Be careful when using async/await in place of Action<>.
They fail silently.
Async void methods considered harmful.
I have a console application using Rx.NET.
I need to block until an IConnectableObservable has completed, for which I am using IConnectableObservable.Wait().
When an unhandled exception is thrown, it is swallowed and the application hangs. I want the application to crash, and the stack track to be printed to the console.
I do not want to add an OnError handler to my IConnectableObserver because this loses the original stack trace.
I have tried using the .Wait() method on the unpublished observable, but this re-subscribes which causes undesirable behaviour.
I have tried using .GetAwaiter().GetResult() instead, but this has the same problem.
var connectable = myObservable.Publish();
connectable.Subscribe(myObserver1);
connectable.Subscribe(myObserver2);
connectable.Connect();
connectcable.Wait();
How can I wait for an IConnectableObservable to complete while retaining typical unhandled exception behaviour?
There's some misdirection in the chain of events here.
The error isn't being swallowed - far from it, it's being re-thrown.
The usual suspects are some weird concurrency and scheduling issues, but nobody suspects the Subscribe method.
When you call Subscribe with something other than your own IObserver<T>, you're creating an AnonymousObserver with these default actions.
new AnonymousObserver<T>(Ignore, Throw, Nop)
which is effectively
new AnonymousObserver<T>(_ => {}, exn => throw exn, () => {})
The default error handler will throw the error on whatever context you're observing on. Yikes. Sometimes it might be the AppDomain timer, or on a pooled thread, and since it can't be handled, your application goes down.
So if we change the sample to provide in a dummy handler,
var myObservable = Observable.Interval(TimeSpan.FromMilliseconds(100)).Take(4).Concat(Observable.Throw(new Exception(), 1L));
var connectable = myObservable.Publish();
connectable.Subscribe(Console.WriteLine, exn => Console.WriteLine("Handled"));
connectable.Subscribe(Console.WriteLine, exn => Console.WriteLine("Handled"));
connectable.Connect();
try
{
connectable.Wait();
}
catch (Exception)
{
Console.WriteLine("An error, but I'm safe");
}
You can handle the error in the Wait like you'd expect to.
Update:
The below behavior only occurs when using the Debug configuration (not optimizing the code). When I change the configuration to Release or tick the 'Optimize code' checkbox in the Build properties, it works just fine.
I'm trying to catch exceptions which occur within a task using Task.ContinueWith as explained in this answer, but the exceptions are not getting handled. Here's a screenshot.
You can reproduce using the following code:
var task = Task.Factory.StartNew(() => { throw new Exception("Oops"); });
task.ContinueWith(t => { Console.WriteLine(t.Exception.Message); },
TaskContinuationOptions.OnlyOnFaulted);
I've also tried the following:
var task = Task.Factory.StartNew(() => { throw new Exception("Oops"); });
task.ContinueWith(t =>
{
if (task.IsFaulted) Console.WriteLine(task.Exception.Message);
});
Any idea why the exception isn't handled?
To expand upon our discussion in the comments:
What you have here is a user-unhandled exception (not to be confused with unobserved exception) causing the debugger to break. If you were to run the program built in debug without actually attaching a debugger, it should behave exactly as you expect. The continuation will run, and it will observe the exception from the antecedent task.
From your perspective, you are handling the exception, and if you were to write some vanilla synchronous code such as this:
try
{
throw new Exception("Oops");
}
catch
{
}
... then the debugger is smart enough to work out that the exception is, indeed, handled, and treats it as such.
However, when you're dealing with task continuations, there are no similarly strong guarantees that your exception handling code will run. It can run on the same thread or a different thread, synchronously or asynchronously, or even not at all if the continuation fails to run (which can happen for a number of reasons that the runtime doesn't necessarily have control over). So the safe choice is for the debugger to say "I can't see this exception being handled anywhere in the immediate call stack, therefore it's user-unhandled".
To drive this point home, think about unobserved task exceptions. In .NET 4.0 they could tear down your program a solid minute after actually being thrown. That is how long it took for the runtime to work out with reasonable confidence that no task continuations actually looked at the exception. When you're debugging, you can't wait that long. If something looks unhandled, the safe choice is for the debugger to break immediately.
Finally, I should note that you can modify this behaviour by telling Visual Studio's debugger not to break on particular exception types (OperationCanceledException would be a good candidate as it comes up in asynchronous code a lot) via Debug -> Windows -> Exception Settings.
A Task that has Continuation that throws an exception will have the Exception wrapped as an AggregateException. If you check the InnerException you will see it contains "Oops"
See Exception Handling in TPL for more information
I have an application that executes several consequent HTTP requests to RESTful API for each of the different items.
The code I have to handle exceptions from executing these requests is similar to the one described in Concurrency in C# Cookbook by Stephen Cleary, recipe 2.4:
var tasks = new List<Task>();
foreach(var item in items)
{
tasks.Add(ProcessItemAsync(item));
}
var parentTask = Task.WhenAll(tasks);
try {
await parentTask;
} catch {
var exceptions = parentTask.Exception;
if (exceptions != null) {
// log individual exceptions
}
}
While analyzing the logs, I noticed that some of the processing did not result in sending all of the needed requests. However, there were no exceptions recorded in the logs.
When debugging the application, I found that calling one of the endpoints resulted in TaskCanceledException being thrown due to a timeout from Web API. However, parentTask.Exception property was null, hence, I did not correctly record this exception in the log.
My questions are the following:
Why isn't TaskCanceledException stored in Task.Exception property?
Are there any other exceptions that would not be stored in Task.Exception property similar to TaskCanceledException?
You can get some more info about this by reading this documentation about task cancellation, especially this part:
If you are waiting on a Task that transitions to the Canceled state, a
System.Threading.Tasks.TaskCanceledException exception (wrapped in an
AggregateException exception) is thrown. Note that this exception
indicates successful cancellation instead of a faulty situation.
Therefore, the task's Exception property returns null.
So main point is - task cancellation is an expected thing, it's not a fault. So no reason to set Exception property - information about cancellation is already recorded in Task state.
Another story is timeout is a fault and should not result in task cancellation. On the other hand, pending web request is cancelled after some time (after timeout), so that's arguable.
As a workaround, you might inspect all tasks in your tasks array to see if they are in cancelled state (IsCancelled returns true) and log accordingly.
TaskCanceledException is specially handled by Task.
When TaskCanceledException is not handled within a Task. It simply sets IsCanceled property within the task to true.
Note that if the Task is awaited TaskCanceledException exception will be thrown, while Task.Exception will remain null.
While it's possible to handle aggregate exceptions, I think it's cleaner to handle them individually:
var tasks = new List<Task>();
foreach(var item in items)
tasks.Add(ProcessItemAndLogExceptionsAsync(item));
await Task.WhenAll(tasks);
private static async Task ProcessItemAndLogExceptionsAsync(Item item)
{
try
{
await ProcessItemAsync(item);
}
catch (Exception ex)
{
// Log exception
}
}
This way you'll get the OperationCanceledExceptions, too.
When I try to raise transient exception manually, it is always handled as AggregateException. Since it is handled as AggregateException , it is not handled as transient error in my retry policy and not retried for the predefined retry count.
Transient errors are shown here .
Therefore I have tried CommunicationException and ServerErrorException but it is handled as an AggregateException.
When I look for AggregateException, it says "Represents one or more errors that occur during application execution." Yeah, it is so helpful!!!
Here is the example code of my case:
I have a retry policy which uses ServiceBusTransientErrorDetectionStrategy
public void TestManually()
{
var retryPolicy = new RetryPolicy<ServiceBusTransientErrorDetectionStrategy>(RetryStrategy.DefaultFixed);
retryPolicy.Retrying += (obj, eventArgs) =>
{
Trace.TraceError("Hey!! I'm Retrying, CurrentRetryCount = {0} , Exception = {1}", eventArgs.CurrentRetryCount, eventArgs.LastException.Message);
};
retryPolicy.ExecuteAsync(() =>
MyTestFunction().ContinueWith(t =>
{
if (t.Exception != null)
{
// A non-transient exception occurred or retry limit has been reached
Trace.TraceError("This was not a transient exxception... It was: " + t.Exception.GetType().ToString());
}
}));
}
public Task MyTestFunction()
{
Task task = Task.Factory.StartNew(() => RaiseTransientErrorManually());
return task;
}
public void RaiseTransientErrorManually()
{
//throw new CommunicationException();
throw new ServerErrorException();
}
Let's say I call my function like this:
TestManually();
I'm very confused why the manually thrown exception (which is defined as Transient Error) is handled as AggregateException ? What I'm missing there?
Thanks.
Exceptions within asynchronous code are a tricky subject for two reasons.
The manner in which exceptions are handled (e.g. by catch blocks) is not always intuitive, and may seem inconsistent.
The manner in which libraries document the behavior for exceptions thrown by asynchronous methods is not always obvious.
I'll address each of these items below.
Important Note: This answer uses the term asynchronous method to refer to any method with a return type of Task or Task<T>. Languages with built-in support for asynchronous programming have their own related terminology which may differ in meaning.
Exceptions Thrown by Asynchronous Methods
Asynchronous methods are capable of throwing exceptions before creating a Task or during the asynchronous execution of the task itself. While projects are not always consistent in the way exceptions are documented for asynchronous code, I like to include the following note with my projects to make things clear for my users.
Note: only assume the following quote is true for a library which explicitly states it. The statement is specifically meant to address the second problem area described above.
The documentation for asynchronous methods does not distinguish between these two cases, allowing for any of the specified exceptions to be thrown in either manner.
Exceptions Prior to Task Creation
Exceptions thrown prior to the creation of the Task object representing the asynchronous operation must be caught directly by the calling code. For example, if the code throws an ArgumentNullException in this manner, the calling code would need to contain an exception handler for ArgumentNullException or ArgumentException to handle the exception.
Example code which throws a direct exception:
public Task SomeOperationAsync()
{
throw new ArgumentException("Directly thrown.");
}
Example code which handles a directly-thrown exception:
try
{
Task myTask = SomeOperationAsync();
}
catch (ArgumentException ex)
{
// ex was thrown directly by SomeOperationAsync. This cannot occur if
// SomeOperationAsync is an async function (§10.15 - C# Language Specification
// Version 5.0).
}
Exceptions During Task Execution
Exceptions thrown during the asynchronous execution of the task are wrapped in an AggregateException object and returned by the Exception property. Exceptions thrown in this manner must be handled either by a task continuation that checks the Exception property, or by calling Wait or checking the Result property within an exception handling block that includes a handler for AggregateException.
In libraries that I create, I provide an additional guarantee for users which reads as follows:
Note: only assume the following quote is true for a library which explicitly states it.
This library additionally ensures that exceptions thrown by asynchronous operations are not wrapped in multiple layers of AggregateException. In other words, an ArgumentException thrown during the asynchronous execution of a task will result in the Exception property returning an AggregateException, and that exception will not contain any nested instances of AggregateException in the InnerExceptions collection. In most cases, the AggregateException wraps exactly one inner exception, which is the original ArgumentException. This guarantee simplifies the use of the API is languages that support async/await, since those operators automatically unwrap the first layer of AggregateException.
Example methods which each throw an exception during task execution:
public Task SomeOperationAsync()
{
return Task.StartNew(
() =>
{
throw new ArgumentException("Directly thrown.");
});
}
public async Task SomeOtherOperationAsync()
{
throw new ArgumentException("async functions never throw exceptions directly.");
}
Example code which handles an exception during task execution:
try
{
Task myTask = SomeOperationAsync();
myTask.Wait();
}
catch (AggregateException wrapperEx)
{
ArgumentException ex = wrapperEx.InnerException as ArgumentException;
if (ex == null)
throw;
// ex was thrown during the asynchronous portion of SomeOperationAsync. This is
// always the case if SomeOperationAsync is an async function (§10.15 - C#
// Language Specification Version 5.0).
}
Consistent Exception Handling
Applications implementing specialized handling for exception which occur during asynchronous calls have multiple options available for consistent handling. The simplest solution, when available, involves using async/await. These operators automatically unwrap the first exception instance in the InnerExceptions collection of an AggregateException, resulting in behavior that appears to calling code as though the exception was directly thrown by the invoked method. The second method involves treating the original call as a continuation of another task, ensuring that all exceptions are presented as an AggregateException to the exception handling code. The following code shows the application of this strategy to an existing asynchronous call. Note that the CompletedTask class and Then() extension method are part of the separate Rackspace Threading Library (open-source, Apache 2.0).
// original asynchronous method invocation
Task task1 = SomeOperationAsync();
// method invocation treated as a continuation
Task task2 = CompletedTask.Default.Then(_ => SomeOperationAsync());
Code using the continuation strategy for consistent error handling may benefit from the use of the Catch() methods, which are also part of the Rackspace Threading Library. This extension method behaves in a manner similar to await, automatically unwrapping the first exception instance in the InnerExceptions collection of an AggregateException before invoking the continuation function which handles the exception.
As of Transient Fault Handling v6.0.1304.0 , the following code successfully retries as per the configured detection strategy:
Strategy:
public class SimpleHandlerStartegy : ITransientErrorDetectionStrategy
{
public bool IsTransient(Exception ex)
{
if (ex is WebException)
{
return true;
}
return false;
}
}
Code That Throws WebException:
async Task<int> SomeAsyncWork()
{
await Task.Delay(1000);
throw new WebException("This is fake");
return 1; // Unreachable!!
}
Client Code:
var retryStrategy = new Incremental(3, TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(2));
var retryPolicy = new RetryPolicy<SimpleHandlerStartegy>(retryStrategy);
retryPolicy.Retrying += (sender, retryArgs) =>
{
Console.WriteLine("Retrying {0}, Delay{1}, Last Exception: {2}", retryArgs.CurrentRetryCount, retryArgs.Delay, retryArgs.LastException);
};
// In real world, await this to get the return value
retryPolicy.ExecuteAsync(() => SomeAsyncWorkThatThrows());
As far as I understand, exceptions that are raised within an asynchronous code block are delivered back to the main thread within an aggregate exception. I suppose this is because raising an exception doesn't necessarily cause execution to be returned to the main thread and therefore we could have more than one exception returned.