We Keep Coding

Intercept HttpClient with third party extensions using state

Injecting state into your HttpRequest when using IHttpClientFactory is achievable by populating HttpRequestMessage.Properties see Using DelegatingHandler with custom data on HttpClient
Now if I have third party extensions on HttpClient (such as IdentityModel), how would I intercept these http requests using custom state?
public async Task DoEnquiry(IHttpClientFactory factory)
{
var id = Database.InsertEnquiry();
var httpClient = factory.CreateClient();
// GetDiscoveryDocumentAsync is a third party extension method on HttpClient
// I therefore cannot inject or alter the request message to be handled by the InterceptorHandler
var discovery = await httpClient.GetDiscoveryDocumentAsync();
// I want id to be associated with any request / response GetDiscoveryDocumentAsync is making
}
The only plausible solution I currently have is to override HttpClient.
public class InspectorHttpClient: HttpClient
{
private readonly HttpClient _internal;
private readonly int _id;
public const string Key = "insepctor";
public InspectorHttpClient(HttpClient #internal, int id)
{
_internal = #internal;
_id = id;
}
public override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
// attach data into HttpRequestMessage for the delegate handler
request.Properties.Add(Key, _id);
return _internal.SendAsync(request, cancellationToken);
}
// override all methods forwarding to _internal
}
A then I'm able to intercept these requests.
public async Task DoEnquiry(IHttpClientFactory factory)
{
var id = Database.InsertEnquiry();
var httpClient = new InspectorHttpClient(factory.CreateClient(), id);
var discovery = await httpClient.GetDiscoveryDocumentAsync();
}
Is that a plausible solution? Something tell me now not to override HttpClient. Quoting from https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpclient?view=net-5.0
The HttpClient also acts as a base class for more specific HTTP clients. An example would be a FacebookHttpClient providing additional methods specific to a Facebook web service (a GetFriends method, for instance). Derived classes should not override the virtual methods on the class. Instead, use a constructor overload that accepts HttpMessageHandler to configure any pre- or post-request processing instead.

I almost included this in my other answer as an alternative solution, but I figured it was too long already. :)
The technique is practically the same, but instead of HttpRequestMessage.Properties, use AsyncLocal<T>. "Async local" is kind of like thread-local storage but for a specific asynchronous code block.
There are a few caveats to using AsyncLocal<T> that aren't particularly well-documented:
Use an immutable nullable type for T.
When setting the async local value, return an IDisposable that resets it.
If you don't do this, then only set the async local value from an async method.
You don't have to follow these guidelines, but they will make your life much easier.
With that out of the way, the solution is similar to the last one, except it just uses AsyncLocal<T> instead. Starting with the helper methods:
public static class AmbientContext
{
public static IDisposable SetId(int id)
{
var oldValue = AmbientId.Value;
AmbientId.Value = id;
// The following line uses Nito.Disposables; feel free to write your own.
return Disposable.Create(() => AmbientId.Value = oldValue);
}
public static int? TryGetId() => AmbientId.Value;
private static readonly AsyncLocal<int?> AmbientId = new AsyncLocal<int?>();
}
Then the calling code is updated to set the ambient value:
public async Task DoEnquiry(IHttpClientFactory factory)
{
var id = Database.InsertEnquiry();
using (AmbientContext.SetId(id))
{
var httpClient = factory.CreateClient();
var discovery = await httpClient.GetDiscoveryDocumentAsync();
}
}
Note that there is an explicit scope for that ambient id value. Any code within that scope can get the id by calling AmbientContext.TryGetId. Using this pattern ensures that this is true for any code: synchronous, async, ConfigureAwait(false), whatever - all code within that scope can get the id value. Including your custom handler:
public class HttpClientInterceptor : DelegatingHandler
{
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var id = AmbientContext.TryGetId();
if (id == null)
throw new InvalidOperationException("The caller must set an ambient id.");
// associate the id with this request
Database.InsertEnquiry(id.Value, request);
return await base.SendAsync(request, cancellationToken);
}
}
Followup readings:
Blog post on "async local" - written before AsyncLocal<T> existed, but has details on how it works. This answers the questions "why should T be immutable?" and "if I don't use IDisposable, why do I have to set the value from an async method?".

Multiple httpclient instances with same implementation in dotnet core

I would like to call a third party API which provided us two different authorization token values. So we are getting two different sets of results back via invoking the same endpoint URL.
EndpointUrl: https://mail.yahoo.com/
Authorization: Token User123
//Do something with the response for User123
Authorization: Token User345
//Do something with the response for User345
In my client service, my wrapper function should invoke this API by calling it twice with different token values. Get the result and merge it.
Here is my service.
public class MailService : IMailService
{
private readonly HttpClient _httpClient;
public MailService(HttpClient httpClient)
{
_httpClient = httpClient;
}
public async Task<UserResponse> GetUserResponse()
{
var uri = new Uri(_httpClient.BaseAddress.AbsoluteUri + "/user-data/");
var response = await _httpClient.GetAsync(uri);
return response;
}
}
I was using Typed Client:
services.AddHttpClient<IMailService,MailService>(client =>
{
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Token", "User123");
client.BaseAddress = new Uri("https://mail.yahoo.com/");
})
Problem Statement:
What is the best way to retrieve the results? I am not sure if I should create two separate instances of HttpClient? I would like to avoid repetitive code if I go with two different classes containing their own HttpClient.
Maybe my solution lies somewhere in Named Client. I just don't know how to implement that gracefully.
Any help would be appreciated.

You can delegate adding the token header later for each message. Remove the auth header from Startup and add Http message handler (Create a new class "AuthHandler").
builder.Services.AddScoped<AuthHandler>();
services.AddHttpClient<IMailService,MailService>(client => {
client.BaseAddress = new Uri("https://mail.yahoo.com/");
})
.AddHttpMessageHandler<AuthHandler>();
In the AuthHandler, you can add the logic to retrieve and set the auth header.
The override SendAsync method will be called every time a http call is made. Below is a sample code, you can modify as per your logic/requirements:
public class AuthHandler : DelegatingHandler
{
private readonly AppSettings _appSettings;
private static string? _accessToken;
public AuthHandler(IOptions<AppSettings> options)
{
_appSettings = options.Value;
}
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request, CancellationToken cancellationToken)
{
var token = GetToken();
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);
return base.SendAsync(request, cancellationToken);
}
private string GetToken()
{
if (_accessToken != null)
{
return _accessToken;
}
_accessToken = "123";// Get your token
return _accessToken;
}
}

What is the best way to retrieve the results?
Named clients are one solution, if you are certain you will always have two authentication headers. That seems very odd to me, though.
SendAsync is another solution. By calling SendAsync instead of GetAsync, you can provide an HttpRequestMessage that can have custom headers. GetAsync is essentially a wrapper around SendAsync for convenience, and you can use the lower-level SendAsync since you need more customization.

I am going to answer, and this is more-or-less opinion based. I like my DI services to be decoupled from everything else in my project. Basically you are putting a configuration to another service in your startup.cs. I like to keep all that stuff in the service that consumes the HttpClient.
So when I inject the IHttpClientFactory, I do it by simply calling:
services.AddHttpClient();
And move on.
Now, in your MailService, you would inject it as so:
public class MailService : IMailService
{
// or store this in your applications external configuration
const Uri BaseUri = new Uri("https://mail.yahoo.com/");
const string UserDataPath = "/user-data/";
private readonly IHttpClientFactory _httpClientFactory;
public MailService(IHttpClientFactory httpClientFactory)
{
_httpClientFactory = httpClientFactory;
}
public async Task<UserResponse> GetUserResponse(string token)
{
var client = _httpClientFactory.CreateClient();
var uri = new UriBuilder(BaseUri){ Path = UserDataPath }.Uri;
using (var msg = new HttpRequestMessage(HttpMethod.Get, uri))
{
msg.Headers.Authorization = new AuthenticationHeaderValue("Token", token);
using (var resp = await _client.SendAsync(msg).ConfigureAwait(false))
{
resp.EnsureSuccessStatusCode();
return await resp.Content.ReadAsAsync<UserResponse>().ConfigureAwait(false);
}
}
}
}
Keep in mind that you should use the HttpRequestMessage pattern and not modify any Default... properties of an HttpClient. It could be in use some where else in your application at that time.
Using HttpRequestMessage ensures you get a fresh set of headers that no one else can modify. To sum it up: HttpClient.Headers are not thread-safe.
ETA:
Now that I think about it... that really is your question. You want to use two different headers using one HttpClient. It won't work in the scenario you presented. You would have to have two HttpClient with their own distinct headers.
So, if you don't want to do it the old-school way I presented, you should consider "Named" clients... each configuration has a different name.

How to create named client after startup?

For my application i need to make a named client for HttpRequests. I can create a named client in Startup. And to access it i inject an "IHttpClientFactory" and create a client from that. But the client needs to have an access token as an authorization header, and i cannot create the token in Startup. Therefor i need a way to create a named client outside of the Startup class. i have already tried injecting "IServiceCollection" into a controller. But this does not work.
Or is there maybe a way to edit a named client after it is already created in startup?

A similar solution to the one posted by #Ruben-J is to create a custom HttpMessageHandler which assigns an authorization header to requests made through the HttpClient at request-time.
You can create a custom HttpMessageHandler that can be assigned to a named HttpClient in Startup like so:
public class YourHttpMessageHandler : DelegatingHandler
{
private readonly IYourTokenProviderService _yourTokenProviderService;
public YourHttpMessageHandler(IYourTokenProviderService yourTokenProviderService)
: base()
{
_yourTokenProviderService = yourTokenProviderService;
}
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var response = SendAsyncWithAuthToken(request, cancellationToken);
if (response.StatusCode == HttpStatusCode.Unauthorized)
{
await _yourTokenProviderService.RefreshTokenAsync();
response = SendAsyncWithAuthToken(request, cancellationToken);
}
return response;
}
private async Task<HttpResponseMessage> SendWithAuthTokenAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", _yourTokenProviderService.Token);
return await base.SendAsync(request, cancellationToken);
}
}
You then configure your services and named HttpClient in Startup:
public virtual void ConfigureServices(IServiceCollection services)
{
...
services.AddTransient<IYourTokenProviderService, YourTokenProviderService>();
services.AddTransient<YourHttpMessageHandler>();
services.AddHttpClient<IYourNamedHttpClient, YourNamedHttpClient>()
.AddHttpMessageHandler<YourHttpMessageHandler>();
...
}
Its worth noting that the current implementation of Polly's AddPolicyHandler is also adding its own DelegatingHandler.
For more background see the Microsoft documentation on adding DelegatingHandler's. Here is also great series of articles from Steve Gordon.

You could use Polly to add a policy handler to your client. You can then add logic if a request returns a 401 Unauthorized. So for example get your service that uses the client to refresh a bearer token and also set it for the current request. This is just a quick solution and maybe there are more elegant solutions. But this will also come in handy if your token expires. Cause then it will be refreshed automatically.
services.AddHttpClient("YourClient")
.AddPolicyHandler((provider, request) =>
{
return Policy.HandleResult<HttpResponseMessage>(r => r.StatusCode == HttpStatusCode.Unauthorized)
.RetryAsync(1, async (response, retryCount, context) =>
{
var service = provider.GetRequiredService<IYourService>();
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", await service.RefreshToken());
});
});

Refresh Token using Polly with Named Client

I have a policy that looks like this
var retryPolicy = Policy
.Handle<HttpRequestException>()
.OrResult<HttpResponseMessage>(resp => resp.StatusCode == HttpStatusCode.Unauthorized)
.WaitAndRetryAsync(3,
retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)),
onRetry: (resp, timeSpan, context) =>
{
// not sure what to put here
});
Then I have a named client that looks like this
services.AddHttpClient("MyClient", client =>
{
client.BaseAddress = new Uri("http://some-url.com");
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", authToken);
client.Timeout = 30000;
})
.AddPolicyHandler(retryPolicy);
I need to refresh the bearer token on the http client in the event I receive a 401. So in a perfect world the following code would do exactly what I'm trying to accomplish
var retryPolicy = Policy
.Handle<HttpRequestException>()
.OrResult<HttpResponseMessage>(resp => resp.StatusCode == HttpStatusCode.Unauthorized)
.WaitAndRetryAsync(3,
retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)),
onRetry: (resp, timeSpan, context) =>
{
var newToken = GetNewToken();
//httpClient doesn't exists here so I need to grab it some how
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", newToken);
});
I have read the following articles:
Re-establishing authentication using Retry
Refresh Token using Polly with Typed Client
retry-to-refresh-authorization
and a couple others as well. However, they all seem use policy.ExecuteAsync() which I don't want to use because then I would have to change all the HttpClient calls throughout my solution. I'm trying to find a way to simply add this functionality to every request by only changing code in the StartUp.cs.

TL;DR: You need to define a communication protocol between a RetryPolicy, a DelegatingHandler and a TokenService.
In case of Typed Clients you can explicitly call the ExecuteAsync and use the Context to exchange data between the to-be-decorated method and the onRetry(Async) delegate.
This trick can't be used in a named client situation. What you need to do instead:
Separate out the Token management into a dedicated service
Use a DelegatingHandler to intercept the HttpClient's communication
This sequence diagram depicts the communication between the different components
Token Service
The DTO
public class Token
{
public string Scheme { get; set; }
public string AccessToken { get; set; }
}
The interface
public interface ITokenService
{
Token GetToken();
Task RefreshToken();
}
The dummy implementation
public class TokenService : ITokenService
{
private DateTime lastRefreshed = DateTime.UtcNow;
public Token GetToken()
=> new Token { Scheme = "Bearer", AccessToken = lastRefreshed.ToString("HH:mm:ss")};
public Task RefreshToken()
{
lastRefreshed = DateTime.UtcNow;
return Task.CompletedTask;
}
}
The registration into the DI as Singleton
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<ITokenService, TokenService>();
...
}
Delegating Handler
The custom exception
public class OutdatedTokenException : Exception
{
}
The handler (interceptor)
public class TokenFreshnessHandler : DelegatingHandler
{
private readonly ITokenService tokenService;
public TokenFreshnessHandler(ITokenService service)
{
tokenService = service;
}
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var token = tokenService.GetToken();
request.Headers.Authorization = new AuthenticationHeaderValue(token.Scheme, token.AccessToken);
var response = await base.SendAsync(request, cancellationToken);
if (response.StatusCode == HttpStatusCode.Unauthorized)
{
throw new OutdatedTokenException();
}
return response;
}
}
It retrieves the current token from the TokenService
It sets the authorization header
It executes the base method
It checks the response's status
If 401 then it throws the custom exception
If other than 401 then it returns with the response
The registration into the DI as Transient
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<ITokenService, TokenService>();
services.AddTransient<TokenFreshnessHandler>();
...
}
Retry Policy
The policy definition
public IAsyncPolicy<HttpResponseMessage> GetTokenRefresher(IServiceProvider provider)
{
return Policy<HttpResponseMessage>
.Handle<OutdatedTokenException>()
.RetryAsync(async (_, __) => await provider.GetRequiredService<ITokenService>().RefreshToken());
}
It receives an IServiceProvider to be able to access the TokenService
It performs a single retry if an OutdatedTokenException was thrown
Inside the onRetryAsync delegate it calls the TokenService's RefreshToken method
Putting all things together
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<ITokenService, TokenService>();
services.AddTransient<TokenFreshnessHandler>();
services.AddHttpClient("TestClient")
.AddPolicyHandler((provider, _) => GetTokenRefresher(provider))
.AddHttpMessageHandler<TokenFreshnessHandler>();
...
}
Please bear in mind that the ordering of AddPolicyHandler and AddHttpMessageHandler matters
If you would call the AddHttpMessageHandler first and then the AddPolicyHandler in that case your retry would not be triggered

This post contains an alternative version of my previously suggested solution.
I'm posting this as a separate answer (rather than editing the previous one) because both solutions are viable and the other post is already a lengthy one.
Why do we need an alternative version?
Because the TokenFreshnessHandler has too much responsibility whereas the Retry policy has too few.
If you look at the SendAsync method overridden implementation then you can see that it perform some operation on the request and on the response as well.
If we could make a separation where
the handler deals only with the request
and policy make its assessment on the response
then we would end up with a much cleaner solution (IMHO).
How can we achieve this separation?
If we could use the Polly's Context as an intermediate storage between the retry attempts then we were able to do this separation. Fortunately the Microsoft.Extensions.Http.Polly package defines two extension methods against the HttpRequestMessage:
SetPolicyExecutionContext
GetPolicyExecutionContext
These are under-documented features. On the docs.microsoft I could not even find the related pages. I have only found them under the dotnet-api-docs repo.
These can be useful if we know that the AddPolicyHandler attaches a new Context to the request only if it did not have one already. Unfortunately, this is yet again not documented, so it is an implementation detail which might change in the future. But currently we can rely on this.
How will this change the protocol?
As you see only difference here is the usage of the Context.
How should we change the handler?
public class TokenRetrievalHandler : DelegatingHandler
{
private readonly ITokenService tokenService;
private const string TokenRetrieval = nameof(TokenRetrieval);
private const string TokenKey = nameof(TokenKey);
public TokenRetrievalHandler(ITokenService service)
{
tokenService = service;
}
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var context = request.GetPolicyExecutionContext();
if(context.Count == 0)
{
context = new Context(TokenRetrieval, new Dictionary<string, object> { { TokenKey, tokenService.GetToken() } });
request.SetPolicyExecutionContext(context);
}
var token = (Token)context[TokenKey];
request.Headers.Authorization = new AuthenticationHeaderValue(token.Scheme, token.AccessToken);
return await base.SendAsync(request, cancellationToken);
}
}
I've changed the name of the handler since its responsibilities have changed
Now, the handler's implementation only cares about the request (and does not care about the response)
As it was said previously: the PolicyHttpMessageHandler creates a new Context if there wasn't any
Because of this the GetPolicyExecutionContext does not return null (even for the very first attempt) rather than a Context with an empty context data collection (context.Count == 0)
How should we change the policy?
public IAsyncPolicy<HttpResponseMessage> GetTokenRefresher(IServiceProvider provider, HttpRequestMessage request)
{
return Policy<HttpResponseMessage>
.HandleResult(response => response.StatusCode == HttpStatusCode.Unauthorized)
.RetryAsync(async (_, __) =>
{
await provider.GetRequiredService<ITokenService>().RefreshToken();
request.SetPolicyExecutionContext(new Context());
});
}
Rather than triggering the policy for a custom exception, now it triggers in case of 401 response's status code
The onRetryAsync has been modified in the way that it clears the attached context of the request
The registration code should be adjusted as well
services.AddHttpClient("TestClient")
.AddPolicyHandler((sp, request) => GetTokenRefresher(sp, request))
.AddHttpMessageHandler<TokenRetrievalHandler>()
Now, we should pass to the GetTokenRefresher method not just the IServiceProvider but also HttpRequestMessage as well
Which solution should I use?
This solution offers nicer separation but it relies on an implementation detail
The other solution makes the handler smart whereas the policy dumb

How to generalize login, action, and logout when using Web API HttpClient

I am using HttpClient (aka Web API client) to consume RESTfull services.
Services require session to be established (via login) and then destroyed (via logout) upon each operation. So the call to consume service A looks something like this (pseudocode)
// setup
create auth dictionary authDict
create authenticationContent using FormUrlEndodeContent(authDict)
create cookieContainer
create HttpClientHandler...
create HttpClient
// login
await httpClient.PostAsync(LoginUrl, authenticationContent);
do error checking
// perform Operation A
await httpClient.....post...or...get...
extract data, process it, tranform it, get a cup of coffee, etc, etc
populate OperationAResult
// logout
await httpClient.GetAsync(LogoutUrl);
// return result
return OperationAResult
My question is, how can I easily reuse setup, login, and logout for different operations?
Should I be creating some method that will take in Action<> and if so how do I make sure that operations occur in order?

Probably the easiest way is to just write a wrapper class.
public class MyHttpClient
{
private HttpClient _client = new HttpClient();
private MyHttpClientSetup _setup;
public MyHttpClient(MyHttpClientSetup setup)
{
this._setup = setup;
}
private void HttpLogin()
{
// .. custom login stuff that uses this._setup
}
private void HttpLogout()
{
// .. custom logout stuff that uses this._setup
}
public void Reset()
{
this._client = new HttpClient();
}
// Wrapped Properties from the private HttpClient (1 example)
public Uri BaseAddress
{
get{ return this._client.BaseAddress;}
set{ this._client.BaseAddress = value;}
}
// Wrapped HttpMethods (1 example)
// Extremely poorly written, should be delegated properly
// This is just a bad example not using Task properly
public Task<HttpResponseMessage> DeleteAsync(string requestUri)
{
this.HttpLogin();
Task<HttpResponseMessage> result = this._client.DeleteAsync(requestUri);
this.HttpLogout();
return result;
}
public class MyHttpClientSetup
{
// Properties required for setup;
}
}

You may be able to create a new MessageHandler to handle this stuff for you transparently.
public class ConnectionHandler : DelegatingHandler {
public HttpClient HttpClient {get;set;}
public TestHandler(HttpMessageHandler handler) {
this.InnerHandler = handler;
}
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, System.Threading.CancellationToken cancellationToken)
{
// Do your login stuff here
return base.SendAsync(request, cancellationToken) // Make your actual request
.ContinueWith(t => {
// Do your logout stuff here
}
}
}
Then you can just use a single instance of a HttpClient to do all your requests. To add your handler to the request/response pipeline you just need to create a regular HttpClientHandler, assign it to the InnerHandler property of your DelegatingHandler and then pass your new handler into the constructor of the HttpClient. From that point on, all requests made via the HttpClient will be routed through your ConnnectionHandler.
var connectionHandler = new ConnectionHandler(new HttpClientHandler());
var client = new HttpClient(connectionHandler);
connectionHandler.HttpClient = client;
var response = client.GetAsync("http://example.org/request").Result;
The advantage of using a single HttpClient instance is that you don't have to keep re-specifying the DefaultRequestHeaders. Also, disposing the HttpClient will kill the TCP Connection so the next request will have to re-open it.