I have a .net 6 console app that I configured with polly policies depending on what each service does.
Program.cs
try
{
//other setup code
services
.AddHttpClient<ISubjectData, SubjectData>()
.AddTransientHttpErrorPolicy(ConfigurePolicy);
//other setup code
IAsyncPolicy<HttpResponseMessage> ConfigurePolicy(PolicyBuilder<HttpResponseMessage> policy)
{
try
{
return policy.Or<TaskCanceledException>()
.WaitAndRetryAsync(Backoff.DecorrelatedJitterBackoffV2(TimeSpan.FromSeconds(10), 5));
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
}
}
catch(Exception e)
{
Console.WriteLine(e);
}
The policy is working, however, the program is throwing an unhandled exception which is TaskCanceledException because the HttpClient timed out, which isn't being caught by either catch statements or the policy in ConfigurePolicy.
How can I catch this error, since it is crashing the app?
Also, is there a way to allow Polly to override HTTP client timeout depending on how long it takes to complete all retries?
TL;DR:
You can't catch the exception because you wrapped the policy definition with try- catches.
In case of Polly the policy definition and its execution are separated.
You define a policy (or a combination of policies)
You are decorating some method(s) with the above policy
You are executing the decorated method explicitly or implicitly through the policy.
Whenever you register a typed/named HttpClient and decorate its methods with an IAsyncPolicy<HttpResponseMessage> then you are doing only the first two steps.
Whenever you call a method on the HttpClient, like PostAsync or SendAsync that's where the policy is being executed on your behalf / implicitly (via a special DelegatingHandler).
That's where you can catch the exceptions
ones which is thrown either by the HttpClient (like TaskCancelledException, HttpRequestException, etc.)
or those that are thrown by one of the policies (like TimeOutRejectedException, BrokenCircuitException, etc.)
UPDATE #1
What is confusing to me, is why does a try-catch block that covers my entire Program.cs not catch the exception, but a try-catch block in the scope of the request does? Aren't exceptions propagated back to the original caller? (The original caller would be a method inside my Program.cs file)
Try catch inside the ConfigurePolicy
As I stated above, here you are building the policy, not executing it. So, the try-catch here can be used to detect and resolve misconfiguration.
Try catch inside Program
Yet again the AddHttpClient and AddTransientHttpErrorPolicy are just builder methods. They are not executing any http request on your behalf. So, wrapping this code into try-catch can help you to spot misconfiguration.
Your try-catch is covering those exceptions which are thrown by the Main but not those which are thrown by different threads which are not in the Main.
Related
I'm attempting to log something before retrying a web api call using Polly in a .net core web api.
I know the web api is failing and returning a 503 response code however there's nothing in my console log as part of the retry call. Any ideas why and how to resolve this?
var retryPolicy = Policy
.Handle<HttpRequestException>()
.Or<SocketException>()
.WaitAndRetryAsync(new[]
{
TimeSpan.FromSeconds(2),
TimeSpan.FromSeconds(5),
TimeSpan.FromSeconds(10)
}, (exception, timeSpan, retryCount, context) =>
{
Console.Write("RETRYING - " + DateTime.Now.Second);
});
await retryPolicy.ExecuteAsync(async () =>
{
var serviceReturnLabel = await this.stockTransfersServiceClient.GetPRReturnLabel(ItemSourceType.ReturnLocker);
if (serviceReturnLabel != null && serviceReturnLabel.Accepted)
{
returnLabel = serviceReturnLabel.PRLabel;
}
});
The retry policy exposes a hook where you can wire up a custom code which will be called before the retry penalty. In other words this delegate will be called whenever the policy should be triggered but before the wait between the two attempts.
This hook is called onRetry or onRetryAsync depending whether your policy is sync or async respectively.
Here you can see when will these user defined custom delegates be called:
Sync Retry Policy
Async Retry Policy
So, you have wired up to the right hook.
Now you have to make sure that policy is triggered. You can use Console.Write or some logger to push information from your delegate to the standard output.
Or you can simply set a breakpoint in your anonymous lambda delegate to make sure that it is called during debugging.
If it is not called then you have to check the following:
Are the thrown exception handled?
Is there any exception at all?
From a policy perspective there can be two kinds of exceptions: handled and unhandled. The former can trigger a new attempt if the threshold is not reached yet. The latter won't trigger another attempt rather it will re-throw the original exception. (Reference)
In your case the policy has been setup to trigger either when a HttpRequestException is thrown or when a SocketException. If the thrown exception is none of these then it is considered unhandled from the policy perspective.
Your policy won't be triggered if there was no exception. There is one typical mistake that we have made several times. Let's suppose we expect that the http response should be 200. Whenever is not success then we want to issue a retry. We might utilize the HandleTransientHttpError (Ref) extension. But that extension watches only the 408 and 5xx status codes. So if we receive for example 429 (too many requests) then no retry will happen. We have to explicitly call the EnsureSuccessStatusCode (Ref) method to throw error if the response was not successful.
Im creating API in .NET Core, which consumes WCF Service. Accessing WCF service is realised, by calling any method, getting an exception (Access denied), and than calling LogIn method using cookie returned in header with first call response. Than, after login i want to retry my original call. All exceptions are the same, and only message string is different. Here is my code for one method call:
public async Task<List<scheduleElement>> getSchedule(DateTime start, DateTime end)
{
bool secondTry = false;
while (true)
{
try
{
var data = await _scheduleServiceClient.getScheduleAsync(start, end);
if (data.#return == null) return new List<scheduleElement>();
return data.#return.ToList();
}
catch (Exception e)
{
if (!secondTry && e.Message.StartsWith("Access denied for WebService method:"))
{
var logged = await LogIntoSOAPServices();
if (!logged) throw;
}
else throw;
secondTry = true;
}
}
}
Im using proxies generated with WCF Web Service Reference Provider
This works, but Im looking for a way to globaly handle exceptions and retry logic like this, because im going to have to copy and paste tons of code. I have Exception handler in my API but if i catch this exceptions with it im not able to retry method i originaly called.
A common library for cases like these is Polly;
https://github.com/App-vNext/Polly
Its part of the dotnet foundation i believe and is quite commonly used.
You can handle specific exceptions or results and act on that, e.g.
// Retry once
Policy
.Handle<SomeExceptionType>()
.Retry()
The logic can get quite complex. For webApi's i usually follow this guide from msdn:
https://learn.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/implement-http-call-retries-exponential-backoff-polly
I have the following middleware in my ASP.NET Core application:
public class ExceptionHandlingMiddleware : IMiddleware
{
// Constructor and other stuff...
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
try
{
await next(context);
}
catch (Exception ex)
{
_logger.Error(ex, "An unhandled exception was thrown!");
await HandleExceptionAsync(context, ex);
}
}
private async Task HandleExceptionAsync(HttpContext context, Exception ex)
{
// Handle exception and return a human-readable
// message in a JSON-object in the response...
}
}
Which I add to the pipeline in the Startup.cs
app.UseMiddleware<ExceptionHandlingMiddleware>();
When running the application from Visual Studio the debugger doesn't break when an exception happens, but is just handled in the middleware and execution continues. However, I want the debugger to break at the place the exception is thrown, when it's not caught by anything but this middleware. How do I achieve this?
I actually tried not adding this middleware when running locally. E.g. doing the following in Startup.cs:
// I'm setting env to "none" when running locally
if (!env.IsEnvironment("none"))
{
app.UseMiddleware<ExceptionHandlingMiddleware>();
}
But this seems to just cause the frontend to act differently - understandably as it's not receiving a readable JSON-object, but it makes me think the problem is not my middleware. I'd prefer to keep this middleware active but if that's not possible while getting the debugger to break properly I guess I can live with that.
I've been reading about errors handling in ASP.NET Core and I came across these 2 ways:
UseExceptionHandler("/error")
UseStatusCodePagesWithRedirects("/error/{0}");
I'm wondering what's the difference between the two? Both redirect to an error page so why use one over the other? I even saw some people using them both at the same time.
You are right that both middlewares do provide error pages. However, they have two different use cases which will make it useful to actually use both at the same time in an application. To understand the differences, let’s take a look at how the middlewares actually work internally.
This is essentially what the StatusCodePages middleware does:
// …
await _next(context);
// …
// Do nothing if a response body has already been provided.
if (context.Response.HasStarted
|| context.Response.StatusCode < 400
|| context.Response.StatusCode >= 600
|| context.Response.ContentLength.HasValue
|| !string.IsNullOrEmpty(context.Response.ContentType))
{
return;
}
var statusCodeContext = new StatusCodeContext(context, _options, _next);
await _options.HandleAsync(statusCodeContext);
It executes the pipeline by calling _next and after the call has returned (meaning that all following middlewares have executed), it will inspect the current response: Basically, if there’s an error status code or no content at all, it will execute the status code page, signalizing the HTTP status code.
The ExceptionHandler middleware on the other hand does something very different:
try
{
await _next(context);
}
catch (Exception ex)
{
// …
try
{
// …
await _options.ExceptionHandler(context);
// …
return;
}
catch (Exception ex2)
{
// Suppress secondary exceptions, re-throw the original.
_logger.ErrorHandlerException(ex2);
}
throw; // Re-throw the original if we couldn't handle it
}
This will try to invoke the middleware pipeline and catch any exception it might produce. Then, it will attempt to run the registered exception handler (which, when setting a path basically means to invoke that path internally and return its response).
So to sum this up:
The StatusCodePages middleware will handle non-successful status code responses and allows you to specify e.g. custom error pages for things like a 404 Not Found.
The ExceptionHandler middleware on the other hand will catch unhandled exceptions in your application and allows you to handle those gracefully for the end user.
Both middlewares have different purposes and actually don’t overlap in what they do. So it often makes sense to include both of them, unless you handle these issues differently of course; e.g. an API will probably not need status code pages, but might still want an exception handler that returns a generic failure and logs everything properly.
I've a few web methods that I use to call some external services like the Google Calendar API, obviously these can be extremely brittle.
Unfortunately I now realise that any error thrown on these methods are not causing an exception to bubble up to Global.asax which is where errors are getting logged in this application.
I have seen suggestions to wrap the method in a try/catch, which is a stupid way of doing it as there are a variety of errors that ASP.Net will silently swallow still.
In trying to find a solution I've seen a lot of references to SoapExtension, which is exactly what I want to do but doesn't get fired as I'm returning Json. What I really want is a way to catch the error just like that.
Any pointers appreciated, I still can't understand how the ASP.Net team could have thought that silently swallowing errors like this was a bright idea.
So for example a method like this:
[WebMethod]
[ExceptionHandling] //can I write a handler like this to catch exceptions from JSON webservices?
static public void DeleteItem(string id)
{
var api = new GoogleCalendarAPI(User.InternalUser());
api.DeleteEvent(id);
return "success";
}
There is no equivalent to SoapExtension for JSON WebMethods and having custom errors turned on in your production site will result in a generic error message being returned to the client, no error is ever raised on the server. You cannot circumvent this.
If you inspect the code using something like ILSpy, there is no way to pass a method or class to page WebMethods like SoapExtension. The error is swallowed by ASP.Net as it invokes the web method, the only notification you will get is a HTTP 500 error sent to the client with a total generic error message.
In 4.0, WebMethods get called by this:
// System.Web.Script.Services.RestHandler
internal static void ExecuteWebServiceCall(HttpContext context, WebServiceMethodData methodData)
{
try
{
//snip irrelevant code
RestHandler.InvokeMethod(context, methodData, rawParams);
}
catch (Exception ex)
{
RestHandler.WriteExceptionJsonString(context, ex);
}
}
So if invoking your method throws an error it will call the following code with a statusCode of 500, there's no re-throw in there and nothing else you can pass in called so unless I'm being blind it just gets swallowed silently. Even worse if you've got custom errors turned on, which any sane person will, it'll completely obfuscate the original cause:
// System.Web.Script.Services.RestHandler
internal static void WriteExceptionJsonString(HttpContext context, Exception ex, int statusCode)
{
//snip code setting up response
context.Response.TrySkipIisCustomErrors = true;
using (StreamWriter streamWriter = new StreamWriter(context.Response.OutputStream, new UTF8Encoding(false)))
{
if (ex is TargetInvocationException)
{
ex = ex.InnerException;
}
if (context.IsCustomErrorEnabled)
{
streamWriter.Write(JavaScriptSerializer.SerializeInternal(RestHandler.BuildWebServiceError(AtlasWeb.WebService_Error, string.Empty, string.Empty)));
}
else
{
streamWriter.Write(JavaScriptSerializer.SerializeInternal(RestHandler.BuildWebServiceError(ex.Message, ex.StackTrace, ex.GetType().FullName)));
}
streamWriter.Flush();
}
}
I can't see a way around it, looks like WebMethod is not ready for production code, shame.
It's not so much they get disappeared, it's more that they get passed out to the calling client. Since however you don't always want to (or should) reveal such intimate details of your service, you can prevent errors bubbling out of your service. This gives the impression of them disappearing.
Wrapping the inner detail in a try-catch is about the best way to cope with any errors. Within the method you're dealing with standard error trapping. So I think you'd want something like:
[WebMethod]
static public string DeleteItem(string id)
{
try
{
var api = new GoogleCalendarAPI(User.InternalUser());
api.DeleteEvent(id);
return "success";
}
catch(Exception ex)
{
log.fatal(ex);
return "error";
}
}
If anything throws an exception within the try-catch it'll be caught. ASP.Net won't interfere with it, unless the methods you are calling have been specifically coded to do so.
Edit
If the GoogleCalendarAPI class is in turn calling a method, such as ExecuteWebServiceCall with catches the Exception, then you'd have to parse the response. I'd hope they gave you some other clue, like a response code, to indicate an error state. You could then wrap that in an Exception, throw it have it caught by your default error handler.