Retry Mechanism catching exception wont work - c#

I am implementing a retry mechanism for an API call to make the same request if the stated business logic occured. I made it really easy for now and will enchance it later with exceptionType handling etc. I simply throw exception if the logic happens and catch them in the retry mechanism but in below the code is just killing the thread and does not execute after the first try. Can you help me please what I am missing here ?
This is the retry logic I am trying to use.
public static T Do<T>(Func<T> action, TimeSpan retryInterval, int attemptCount)
{
var exceptions = new List<Exception>();
for (int attempted = 0; attempted < attemptCount; attempted++)
{
try
{
if (attempted > 0)
{
Thread.Sleep(retryInterval);
}
return action();
}
catch (Exception ex)
{
exceptions.Add(ex);
}
}
throw new AggregateException(exceptions);
}
This is how I invoke the retry method.
await Task.Run(() =>
{
RetryHelper.Do(() => ConfirmRequestRetryAsync(request, true), TimeSpan.FromSeconds(60), 10);
});
And this is the method which can throw exceptions due to logic.
public async void ConfirmRequestRetryAsync(ConfirmRequest request, bool flag)
{
logger.Info($"Confirm Request Async Called for the request : {JsonConvert.SerializeObject(request)}");
var confirmRequest = GetSignedConfirmRequest(request.PaymentId);
var confirmResponse = await MakeRequest(confirmRequest);
//Added flag and sending at false at first try to not throw exception
//Then in retry mechanism this exceptions will be use to trigger retry logic.
if (flag)
{
var statu = ConfirmResponseXmlConvert(confirmResponse);
if (statu.Item1 == "0" && statu.Item2 == "InProcess")
{
throw new Exception("InProcess");
}
else if (statu.Item1 == "-1" && statu.Item2 != "Declined")
{
throw new Exception("Error");
}
}
}

The ConfirmRequestRetryAsync is an async void method. This means it will return at the first await. The rest of the code will run at some later time, in the same thread context as the caller. So when the exceptions are thrown the retrying method has already returned and there is nothing to catch the exceptions.
The fix is to make it an async Task method, and await this in your retry-method. This might require two variants of the retrying method, one for async methods and one for non-async.
A rule of thumb for async void is to never allow exceptions to be thrown from them, since there is no possibility for anyone to catch them. Always use async Task if exceptions is a possibility.

Related

How to keep ActionBlock running on any exception

I have an ActionBlock that simply processes messages that comes from an infinite loop continuously. Inside ActionBlock I make an http post. When on any network related error, the method throws exception and the block is faulted/stopped. This is not the behavior that I want. I want processing runs even though exception occurs. (Keep hitting Process method) To simulate my program;
private static ExecutionDataflowBlockOptions processBlockOptions
{
get
{
return new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 1
};
}
}
static async Start()
{
processQueue = new
ActionBlock<QueueMessage>(
async (item) =>
{
await Process(item);
},
processBlockOptions);
while (!Stopped)
{
//Read from DB and do logic with item
QueueMessage item= new QueueMessage();
await processQueue.SendAsync(item);
}
}
private async static Task<int> Process(QueueMessage item)
{
try
{
await item.HttpPost(order);
}
catch (Exception ex)
{
//Http endpoint might be broken
throw ex;
}
}
You're re-throwing your exceptions, and you're doing it wrong:
throw ex;
If you need to log the errors or stop the pipeline for a while, you don't need to throw anything, simply log ex.ToString() and react accordingly. Second thing is that you should use throw; instead of throw ex;, as you're rewriting the stack trace for exception, which doesn't really matter in this case but can be misleading in case of more complicated workflow.

Task swallows the exception thrown

In the method below, when an exception is thrown in the TRY block, it is being swallowed. How can I make it throw the exception so that it gets written to log in the catch block? The log writer works fine. Thanks!
public static bool MonitorQueueEmptyTask(string queueName, CancellationTokenSource tokenSource)
{
try
{
Task<bool> task = Task.Factory.StartNew<bool>(() =>
{
while (!QueueManager.IsQueueEmpty(queueName))
{
if (tokenSource.IsCancellationRequested)
{
break;
}
Thread.Sleep(5000);
throw new Exception("Throwing an error!"); //THIS THROW IS SWALLOWED -- NO LOG WRITTEN ON CATCH
};
return true;
}, tokenSource.Token);
}
catch (Exception ex)
{
WriteExceptionToLog(ex.Stack); //it's not that this method doesn't work. it works fine.
return false;
}
return true;
}
If you want to fire and forget, you can attach a continuation using ContinueWith. The current try-catch will not help you at all, as the exception is encapsulated inside the Task. If this is "fire and forget", than you can log the exception:
public static Task MonitorQueueEmptyTask(
string queueName, CancellationTokenSource tokenSource)
{
return Task.Factory.StartNew<bool>(() =>
{
while (!QueueManager.IsQueueEmpty(queueName))
{
if (tokenSource.IsCancellationRequested)
{
break;
}
Thread.Sleep(5000);
throw new Exception("Throwing an error!");
};
}, tokenSource.Token, TaskCreationOptions.LongRunning).ContinueWith(faultedTask =>
{
WriteExceptionToLog(faultedTask.Exception);
}, TaskContinuationOptions.OnlyOnFaulted);
}
This, in turn, will not propagate the exception after it's thrown, but will provide a mechanism to log the error. If you want the exception to be properly handled, you can register to TaskScheduler.UnobservedTaskException. Additionally, you can set ThrowUnobservedTaskExceptions enabled="true" in your configuration if you want unhandled exceptions to terminate your application. ContinueWith will consider the exception "handled" once you look at the task.Exception property.
The exception is not swallowed; it's just that it doesn't occur on the thread that executes the try/catch block, but on the separate Task thread.
If you don't observe the task's result or exception, when the task is eventually garbage collected, it will throw an exception saying that the task was not observed. Unless you catch that by handling the TaskScheduler.UnobservedTaskException, it will crash the process.
I also had a problem with this, and i really dislike the whole idea of App.config, so can provide another solution to prevent the exceptions disappearing :)
Save the exception then throw it after the Task.Run has completed, e.g.
private async void Function() {
Exception save_exception = null;
await Task.Run(() => {
try {
// Do Stuff
} catch (Exception ex) {
save_exception = ex;
}
}).ContinueWith(new Action<Task>(task => {
if (save_exception != null)
throw save_exception;
// Do Stuff
}));
}

Is there a way to Wait for a TPL Task without in throwing an exception?

Some of us prefer to code in an exception-light style. However, if you wait for a Task Parallel Library task, and the task threw an exception, it will throw an exception on the calling thread as well. Is there a (preferably standard) way to avoid this behaviour and just check the response for exceptions when you get it back?
You can use Task.WaitAny like:
var task = Task.Run(() =>
{
// ...
throw new Exception("Blah");
});
Task.WaitAny(task);
if (task.IsFaulted)
{
var error = task.Exception;
// ...
}
else if (task.IsCanceled)
{
// ...
}
else
{
// Success
}
You can't wait on a faulted task without raising an exception.
But you can wait on a continuation to that task, which will complete only after the original task completed without raising an exception:
public static Task SwallowExceptions(this Task task)
{
return task.ContinueWith(_ => { });
}
faultedTask.SwallowExceptions().Wait();
if (faultedTask.IsFaulted)
{
// handle exception
}
If your task returns a value, you can represent that in the extensions method and return the actual value if there were no exceptions or the default value if there were:
public static Task<T> SwallowExceptions<T>(this Task<T> task)
{
return task.ContinueWith(completedTask =>
completedTask.IsFaulted
? default(T)
: completedTask.Result);
}
Unfortunately, this functionality is not built-in. Use this workaround:
myTask.ContinueWith(_ => { }, TaskContinuationOptions.ExecuteSynchronously).Wait();
You can make this into an extension method.
Based on what you have written, might catching the exception and checking the IsFaulted property be your solution? IsFaulted
Catch the exception within the Task and return it in the result?
var task1 = Task.Factory.StartNew(() =>
{
try
{
throw new MyCustomException("I'm bad, but not too bad!");
}
catch(Exception ex)
{
return new Result { Error = ex };
}
});

Restarting a task in the background if certain errors occur

I am using some REST requests using Mono.Mac (3.2.3) to communicate with a server, and as a retry mechanism I am quietly attempting to give the HTTP actions multiple tries if they fail, or time out.
I have the following;
var tries = 0;
while (tries <= ALLOWED_TRIES)
{
try
{
postTask.Start();
tries++;
if (!postTask.Wait(Timeout))
{
throw new TimeoutException("Operation timed out");
}
break;
} catch (Exception e) {
if (tries > ALLOWED_TRIES)
{
throw new Exception("Failed to access Resource.", e);
}
}
}
Where the task uses parameters of the parent method like so;
var postTask = new Task<HttpWebResponse>(() => {return someStuff(foo, bar);},
Task.Factory.CancellationToken,
Task.Factory.CreationOptions);
The problem seems to be that the task does not want to be run again with postTask.Start() after it's first completion (and subsequent failure). Is there a simple way of doing this, or am I misusing tasks in this way? Is there some sort of method that resets the task to its initial state, or am I better off using a factory of some sort?
You're indeed misusing the Task here, for a few reasons:
You cannot run the same task more than once. When it's done, it's done.
It is not recommended to construct a Task object manually, there's Task.Run and Task.Factory.Start for that.
You should not use Task.Run/Task.Factory.Start for a task which does IO-bound work. They are intended for CPU-bound work, as they "borrow" a thread from ThreadPool to execute the task action. Instead, use pure async Task-based APIs for this, which do not need a dedicate thread to complete.
For example, below you can call GetResponseWithRetryAsync from the UI thread and still keep the UI responsive:
async Task<HttpWebResponse> GetResponseWithRetryAsync(string url, int retries)
{
if (retries < 0)
throw new ArgumentOutOfRangeException();
var request = WebRequest.Create(url);
while (true)
{
try
{
var result = await request.GetResponseAsync();
return (HttpWebResponse)result;
}
catch (Exception ex)
{
if (--retries == 0)
throw; // rethrow last error
// otherwise, log the error and retry
Debug.Print("Retrying after error: " + ex.Message);
}
}
}
More reading:
"Task.Factory.StartNew" vs "new Task(...).Start".
Task.Run vs Task.Factory.StartNew.
I would recommend doing something like this:
private int retryCount = 3;
...
public async Task OperationWithBasicRetryAsync()
{
int currentRetry = 0;
for (; ;)
{
try
{
// Calling external service.
await TransientOperationAsync();
// Return or break.
break;
}
catch (Exception ex)
{
Trace.TraceError("Operation Exception");
currentRetry++;
// Check if the exception thrown was a transient exception
// based on the logic in the error detection strategy.
// Determine whether to retry the operation, as well as how
// long to wait, based on the retry strategy.
if (currentRetry > this.retryCount || !IsTransient(ex))
{
// If this is not a transient error
// or we should not retry re-throw the exception.
throw;
}
}
// Wait to retry the operation.
// Consider calculating an exponential delay here and
// using a strategy best suited for the operation and fault.
Await.Task.Delay();
}
}
// Async method that wraps a call to a remote service (details not shown).
private async Task TransientOperationAsync()
{
...
}
This code is from the Retry Pattern Design from Microsoft. You can check it out here: https://msdn.microsoft.com/en-us/library/dn589788.aspx

Handle exception thrown by a task

I have a task running a long time operation in WPF:
Task t = Task.Factory.StartNew(() =>
{
try
{
process(cancelTokenSource.Token, CompressionMethod, OpInfo);
}
catch (OperationCanceledException)
{
logger.Info("Operation cancelled by the user");
}
}, cancelTokenSource.Token);
try
{
t.Wait();
}
catch (AggregateException ae)
{
int i = 0;
}
private void process(CancellationToken token, CompressionLevel level, OperationInfo info)
{
// check hash
if (ComputeHash)
{
logger.Info("HASH CHECKING NOT IMPLEMENTED YET!");
MessageBox.Show(this,"HASH CHECKING NOT IMPLEMENTED YET!", "WARNING", MessageBoxButton.OK, MessageBoxImage.Warning);
}
token.ThrowIfCancellationRequested();
UserMsgPhase = "Operation finished";
return info;
}
Problem is "MessageBox.Show" throws an exception and it is not captured within "catch (AggregateException ae)". I've been reading about TPL exception handling but I don't understand why it is not catched. Please, could you help me?
Once the task is complete you can check its Exception property. You also have Status and IsCompleted properties which may be useful to you...
Check Task.Exception.
If your task is typed (returning a result), then accessing myTask.Result will throw this exception.
Moreover, if you are running .Net 4.5, you could use async/await.
As an example:
public async void MyButton_OnClick(object sender, EventArgs e)
{
try
{
Task t = ...your task...;
var myResult = await t; // do whatever you like with your task's result (if any)
}catch
{
// whatever you need
}
}
as you would do with synchronous code (but this is not an actual synchronous call)
I believe that the question's process method is a Task, so it looks like it could be implement in a different manner:
You can make the process to be implemented as Task and then you will have a task-child within task-parent.
Then you can make use of the TaskCreationOptions.AttachedToParent option.
According to Stephen Toub, using AttachedToParent will help notify children-task exception to the parent-task catch:
any exceptions from faulted children will propagate up to the parent
Task (unless the parent Task observes those exceptions before it
completes).
Example:
I've omitted the cancellation token parts in order for it to be more simple.
Task t = Task.Factory.StartNew(() =>
{
var process = new Task(() =>
{
//Copy here the process logic.
}, TaskCreationOptions.AttachedToParent);
//*Private failure handler*.
process.start();
});
try
{
t.Wait();
}
catch (AggregateException ae)
{
//handle exceptions from process.
}
In addition, you may add a private failure handler like:
//*Private failure handler*.
var failHandler = child.ContinueWith(t =>
{
//Oops, something went wrong...
}, TaskContinuationOptions.AttachedToParent|TaskContinuationOptions.OnlyOnFaulted);

Categories