Asp.NET core logs each request that enters based on configuration. Now i'd like to have the same functionality for Flurl requests i sent. Most notably, I of course would like to know when a requests fails or does not complete. For debugging I found logging all requests in a verbose matter was extremely helpful.
Sure can. For cross-cutting concerns like logging, you want to use Flurl's event handlers, specifically BeforeCall, AfterCall, OnError, and their async equivalents (BeforeCallAsync, AfterCallAsync, OnErrorAsync). Here's an error logging example:
private async Task HandleFlurlErrorAsync(HttpCall call)
{
await LogErrorAsync(call.Exception.Message);
call.ExceptionHandled = true; // prevents exception from bubbling up, if desired
}
// Configure once at startup:
FlurlHttp.Configure(settings => settings.OnErrorAsync = HandleFlurlErrorAsync);
Related
I want to configure a HttpClient to have a default authorization header that results from an async call like this:
builder.Services.AddHttpClient("client", (serviceProvider, client) =>
{
client.DefaultRequestHeaders.Authorization =
serviceProvider.GetService<IAuthProvider>()?.GetAuthHeaderAsync().Result;
});
Is it ok to call Task.Result in configuration methods that do not support async delegates? or should I do this:
builder.Services.AddHttpClient("client", async (serviceProvider, client) =>
{
client.DefaultRequestHeaders.Authorization =
await serviceProvider.GetService<IAuthProvider>()?.GetAuthHeaderAsync();
});
When I do this second option I'm getting a warning: Avoid using
'async' lambda when delegate type returns 'void'
Dependency injection is one place where the "use async all the way" general advice isn't applicable, since no DI/IoC container supports asynchronous resolution.
One way to work around this is to use "asynchronous factory" types; i.e., you define an HttpClient factory that can asynchronously produce a client on demand.
Another approach is to just do DI/IoC injection synchronously, usually blocking on all asynchronous construction at startup time, before the application has actually started.
A final approach is more advanced: you define a type that wraps all the APIs you need (with asynchronous signatures), and you hide the asynchronous initialization behind an async lazy (or a single-item async cache) within that type.
Regarding your specific use case, is the authentication header something that is only requested once, and remains constant for the rest of the app? If so, I would say to just synchronously block on it at startup. If it's something that can change, then you're probably better off handling authentication as a message handler, not at the point of constructing a client.
I am using Polly's retry policy for my unsuccessful call. But it is not catching the exception and retrying.
Using:
Polly 7.2.3
.NET6.0
Nsubstitute 4.2.2
Setup:
var delay = Backoff.DecorrelatedJitterBackoffV2(TimeSpan.FromMilliseconds(RetryDelay), RetryCount);
_retryPolicy = Policy.Handle<HttpRequestException>()
.Or<CustomException>()
.OrResult<string>(response => !string.IsNullOrEmpty(response))
.WaitAndRetryAsync(delay);
Usage:
public async Task ProcessRequest()
{
var errors = await _retryPolicy.ExecuteAsync(async () => await this.RetryProcessRequest());
}
private async Task<string> RetryProcessRequest()
{
var token = await _tokenInfrastructure.GetTokenAsync();
return await _timezoneInfrastructure.ProcessRequest(token);
}
Unit test:
[Fact]
public async Task ProcessRequest_Throws()
{
string errors = _fixture.Create<string>();
var token = _fixture.Create<string>();
// Retry policy configured to retry 3 times on failed call
var expectedReceivedCalls = 4;
// this is throwing but Polly is not catching it and not retrying
_tokenInfrastructure.GetTokenAsync().Returns(Task.FromException<string>(new HttpRequestException()));
// this errors can be caught by Polly as configured and retrying
_timezoneInfrastructure.ProcessRequest(token).Returns(errors);
await _timezoneOrchestration.Awaiting(o => o.ProcessRequest()).Should()
.ThrowAsync<HttpRequestException>();
await _tokenInfrastructure.Received(expectedReceivedCalls).GetTokenAsync();
await _timezoneInfrastructure.Received(expectedReceivedCalls).ProcessRequest(Arg.Any<string>());
}
After doing Rubber duck debugging found my mistake. Actually, Polly was configured well and retrying.
this line of code was never calling because above we were getting exceptions.
return await _timezoneInfrastructure.ProcessRequest(token);
In Unit tests, it was expecting some retry calls:
_timezoneInfrastructure.Received(expectedReceivedCalls).ProcessRequest(Arg.Any<string>());
This post is not answer for the OP's question (the problem has already been addressed here). It is more like a set of suggestions (you can call it code review if you wish).
Exponential backoff
I'm glad to see that you are using the V2 of the backoff logic which utilizes the jitter in a proper way.
My only concern here is that depending on the actual values of RetryDelay and RetryCount the sleepDuration might explode: It can easily reach several minutes. I would suggest two solutions in that case:
Change factor parameter of the DecorrelatedJitterBackoffV2 from 2 (which is the default) to a lower number
Or try to top the max sleepDuration, here I have detailed one way to do that
Combined Retry logic
Without knowing what does RetryProcessRequest do, it seems like this _retryPolicy smashes two different policies into one. I might be wrong, so this section could suggest something which is not applicable for your code.
I assume this part decorates the _tokenInfrastructure.GetTokenAsync call
_retryPolicy = Policy.Handle<HttpRequestException>()
.WaitAndRetryAsync(delay);
whereas this part decorates the _timezoneInfrastructure.ProcessRequest call
_retryPolicy = Policy.Handle<CustomException>()
.OrResult<string>(response => !string.IsNullOrEmpty(response))
.WaitAndRetryAsync(delay);
Based on your naming I assume that these are different downstream systems: tokenInfrastructure, timezoneInfrastructure. I would suggest to create separate policies for them. You might want to apply different Timeout for them or use separate Circuit Breakers.
Naming
I know naming is hard and I assume your method names (ProcessRequest, RetryProcessRequest or ProcessRequest_Throws) are dumyfied for StackOverflow. If not then please try to spend some time to come up with more expressive names.
Component testing
Your ProcessRequest_Throws test is not really a unit test. It is more likely a component test. You are testing there the integration between the Polly's policy and the decorated code.
If you would test only the correctness of the policy setup or test only the decorated code (with NoOpPolicy) then they were unit tests.
I'm attempting to log something before retrying a web api call using Polly in a .net core web api.
I know the web api is failing and returning a 503 response code however there's nothing in my console log as part of the retry call. Any ideas why and how to resolve this?
var retryPolicy = Policy
.Handle<HttpRequestException>()
.Or<SocketException>()
.WaitAndRetryAsync(new[]
{
TimeSpan.FromSeconds(2),
TimeSpan.FromSeconds(5),
TimeSpan.FromSeconds(10)
}, (exception, timeSpan, retryCount, context) =>
{
Console.Write("RETRYING - " + DateTime.Now.Second);
});
await retryPolicy.ExecuteAsync(async () =>
{
var serviceReturnLabel = await this.stockTransfersServiceClient.GetPRReturnLabel(ItemSourceType.ReturnLocker);
if (serviceReturnLabel != null && serviceReturnLabel.Accepted)
{
returnLabel = serviceReturnLabel.PRLabel;
}
});
The retry policy exposes a hook where you can wire up a custom code which will be called before the retry penalty. In other words this delegate will be called whenever the policy should be triggered but before the wait between the two attempts.
This hook is called onRetry or onRetryAsync depending whether your policy is sync or async respectively.
Here you can see when will these user defined custom delegates be called:
Sync Retry Policy
Async Retry Policy
So, you have wired up to the right hook.
Now you have to make sure that policy is triggered. You can use Console.Write or some logger to push information from your delegate to the standard output.
Or you can simply set a breakpoint in your anonymous lambda delegate to make sure that it is called during debugging.
If it is not called then you have to check the following:
Are the thrown exception handled?
Is there any exception at all?
From a policy perspective there can be two kinds of exceptions: handled and unhandled. The former can trigger a new attempt if the threshold is not reached yet. The latter won't trigger another attempt rather it will re-throw the original exception. (Reference)
In your case the policy has been setup to trigger either when a HttpRequestException is thrown or when a SocketException. If the thrown exception is none of these then it is considered unhandled from the policy perspective.
Your policy won't be triggered if there was no exception. There is one typical mistake that we have made several times. Let's suppose we expect that the http response should be 200. Whenever is not success then we want to issue a retry. We might utilize the HandleTransientHttpError (Ref) extension. But that extension watches only the 408 and 5xx status codes. So if we receive for example 429 (too many requests) then no retry will happen. We have to explicitly call the EnsureSuccessStatusCode (Ref) method to throw error if the response was not successful.
I have two applications using Rebus in ASP.NET MVC Core
I am able send messages between two applications using Bus.Send(...). What I can't is to publish event such as CustomerCreated after creating so that other applications can take actions.
I have configured the application as follows
public void ConfigureServices(IServiceCollection services)
{
services.AutoRegisterHandlersFromAssemblyOf<Handler1>();
services.AddRebus(configure => configure
.Logging(l => l.Use(new MSLoggerFactoryAdapter(_loggerFactory)))
.Transport(t=>t.UseRabbitMq("amqp://guest:guest#localhost:5672", "rebus_rabbit_first"))
.Sagas(x => x.StoreInSqlServer("Data Source=.;Initial Catalog=RebusDBRabbit;User ID=student;Password=student;", "Sagas", "SagaIndex"))
.Options(o =>
{
o.SetNumberOfWorkers(10);
o.SetMaxParallelism(20);
o.HandleMessagesInsideTransactionScope();
o.SimpleRetryStrategy(errorQueueAddress: "somewhere_else", maxDeliveryAttempts: 10, secondLevelRetriesEnabled: true);
})
.Routing(r => r.TypeBased()
.MapAssemblyOf<CreateStudent>("rebus_rabbit_second")));
services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
}
In the Controller I Send a message to another Application as follows
CreateStudent student = new CreateStudent { StudentID="90008", StudentName="Amour Rashid Hamad",DateOfBirth=DateTime.Parse("1974-03-18") };
_bus.Send(student).Wait();
This is OK.
Now My Problem is to publish an event to broadcast the event to other interested parties eg
_bus.Publish(new StudentCreated { StudentID="90008",Remarks="Hurray We have been Successfully"});
How Do I Subscribe to the event as per my configuration. I have seen some samples but I could not understand them. Adding to my implementation would be preferred.
In the Service Configuration I did as follows:
app.ApplicationServices.UseRebus(async bus => {
await bus.Subscribe<StudentCreated>();
});
and then created a handler
public class StudentCreatedEventHandler : IHandleMessages<StudentCreated>, IHandleMessages<IFailed<StudentCreated>>
{
readonly IBus _bus;
public StudentCreatedEventHandler(IBus bus)
{
_bus = bus;
}
public async Task Handle(StudentCreated student)
{
// do stuff that can fail here...
var remarks = $"Remarks on RebusWithRabbit1 : {student.Remarks}";
}
public async Task Handle(IFailed<StudentCreated> failedMessage)
{
await _bus.Advanced.TransportMessage.Defer(TimeSpan.FromSeconds(30));
}
}
This could handle the events published.
I just want to get assured if that is the proper way of doing it.
I have however noticed one thing. If I have more than one endpoints Subscribing to the event only one is notified. I expected that multiple endpoints could need to be notified and every one may execute a different process from the same event.
Is there any way to change this behavior. I remember in MassTransit this is the default behavious.
Thanks
It looks like you're using await bus.Send(...) properly.
As you've probably figured out, Send looks up a destination queue and sends to that (and only that), and the lookup is done from the endpoint mappings (where you're currently mapping all message types to the queue named rebus_rabbit_second).
When you want to await bus.Publish(...), you need someone to await bus.Subscribe<T>() accordingly. Underneath the covers, Rebus will use the .NET type as the topic, so if you
await bus.Subscribe<SomeEvent>();
in one application, and
await bus.Publish(new SomeEvent());
in another, your subscriber will receive the event.
TL;DR: You need to
await bus.Subscribe<StudentCreated>();
in the application where you want to receive published events of type StudentCreated.
Btw. you should EITHER use C#'s support for calling asynchronous methods
await bus.Send(yourMessage);
or invoke Rebus' synchronous API to do your work:
var syncBus = bus.Advances.SyncBus;
syncBus.Send(yourMessage); //< returns void
Rebus' synchronous methods do not deadlock the thread, e.g. if you're calling them from ASP.NET or WCF.
I'm pretty new to working with HTTP stuff so I'm rather confused as to what would be the best approach to request data from a HTTP address every few seconds or so.
The API I'm using has - at least to my knowledge no webhook support. So I imagine the way to update my data would be a rather crude way of doing so.
I want this to happen in the background so the GUI does not freeze and become unresponsive. So I know I (probably) need to fiddle with threads.
Best results I've had has been with a async/await Timer. I'm not entirely sure how to work with this and the only way for me to get it to work is to throw an exception after it has elapsed. If I don't - it says that not all nodes return a value and I can't even use return which really, really confuses me.
How should I be doing this?
If it's of any use, I'm working on creating my own RCON tool for a game which has all kinds of server data available via a HTTP API - but documentation for this API is very lackluster.
if you go to .net core you can see my previous answer on: Start multiple background threads inside Self Hosted ASP.NET Core Microservice
for .net framework you have to do a little more yourself. But still very do-able!
in your global.asax you have (or should I say: should) have your dependency injection. Something like:
protected void Application_Start()
{
Bootstrap();
//and do something more
}
private static void Bootstrap()
{
var container = new Container();
container.Register(() => new HttpClient());
container.RegisterSingleton<IApiCaller, ApiCaller>();
container.RegisterInitializer<IApiCaller>(ApiCaller=> apicaller.StartCallingAsync());
// Suppress warnings for HttpClient
var registration = container.GetRegistration(typeof(HttpClient)).Registration;
registration.SuppressDiagnosticWarning(DiagnosticType.DisposableTransientComponent, "Dispose is being called by code.");
registration.SuppressDiagnosticWarning(DiagnosticType.LifestyleMismatch, "Every HttpCient is unique for each dependency.");
container.Verify();
GlobalConfiguration.Configuration.DependencyResolver = new SimpleInjectorWebApiDependencyResolver(container);
}
In this case, I let SimpleInjector start my background thread to do a lot of work.
In the apicaller you can do your httpcalls.
something like:
public async Task StartCallingAsync(CancellationToken cancellationToken = (default)CancellationToken)
{
while(true)
{
var response = await _httpClient.GetAsync(url);
if (response.IsSuccessStatusCode)
{
//do work
}
await Task.Delay(10000, cancellationToken);
}
}
for the GetAsync there are extension methods that can cast it directly to your object.
can you work with this?