I have an old version of ASP.NET MVC app that doesn't have a Startup.cs. I wanted to implement a clean way to have an HttpClient that I would use for my API calls to third parties.
Here's what I've done so far based on some ideas/recommendations I've received for this question. The problem is that when I make the API call, it goes nowhere. I put it in a try catch but I'm not even getting an exception. The API provider tells me that they're not seeing the search parameter.
First, I created this HttpClientAccessor for lazy loading.
public static class HttpClientAccessor
{
public static Func<HttpClient> ValueFactory = () =>
{
var client = new HttpClient();
client.BaseAddress = new Uri("https://apiUrl.com");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.TryAddWithoutValidation("APIAccessToken", "token1");
client.DefaultRequestHeaders.TryAddWithoutValidation("UserToken", "token2");
return client;
};
private static Lazy<HttpClient> client = new Lazy<HttpClient>(ValueFactory);
public static HttpClient HttpClient
{
get { return client.Value; }
}
}
I then created an API client of my own so that I can have the API call functions in one place which looks like this:
public class MyApiClient
{
public async Task GetSomeData()
{
var client = HttpClientAccessor.HttpClient;
try
{
var result = await client.GetStringAsync("somedata/search?text=test");
var output = JObject.Parse(result);
}
catch(Exception e)
{
var error = e.Message;
}
}
}
Then in my ASP.NET Controller action, I do this:
public class MyController : Controller
{
private static readonly MyApiClient _apiClient = new MyApiClient ();
public ActionResult ApiTest()
{
var data = _apiClient.GetSomeData().Wait();
}
}
Any idea where my mistake is?
UPDATE:
This simple approach works fine:
public class MyController : Controller
{
private static readonly HttpClient _client = new HttpClient();
public ActionResult ApiTest()
{
_client.BaseAddress = new Uri("https://apiUrl.com");
_client.DefaultRequestHeaders.Accept.Clear();
_client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
_client.DefaultRequestHeaders.TryAddWithoutValidation("APIAccessToken", "token1");
_client.DefaultRequestHeaders.TryAddWithoutValidation("UserToken", "token2");
var response = _client.GetStringAsync("somedata/search?text=test").Result;
}
}
As mentioned, dependency injection is not being utilized so technically there is no need for a composition root where these things would have been initialized.
If there is no need to actually initialize the client on start up you could consider using a Lazy singleton approach.
An example
public static class HttpClientAccessor {
public static Func<HttpClient> ValueFactory = () => {
var client = new HttpClient();
client.BaseAddress = new Uri("https://apiUrl.com");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.TryAddWithoutValidation("APIAccessToken", "token1");
client.DefaultRequestHeaders.TryAddWithoutValidation("UserToken", "token2");
return client;
};
private static Lazy<HttpClient> client = new Lazy<HttpClient>(ValueFactory);
public static HttpClient HttpClient {
get { return client.Value; }
}
}
The factory delegate of the Lazy<HttpClient> can be made more complex if additional settings are needed on the client.
And where ever the client is needed you call the service
var client = HttpClientAccessor.HttpClient;
var response = await client.GetStringAsync("{url}");
the client will be initialized on first use and you will get the same instance on subsequent calls for the instance.
As used in your controller, you are mixing async calls with blocking calls line .Wait() or .Result. This can lead to deadlocks and should be avoided.
public class MyController : Controller {
private static readonly MyApiClient _apiClient = new MyApiClient ();
public async Task<ActionResult> ApiTest() {
var data = await _apiClient.GetSomeData();
//...
}
}
Code should be async all the way through.
Reference Async/Await - Best Practices in Asynchronous Programming
The Application_Start() method is the right place. But I would have to ask: why you have to create the HttpClient instance when the "application starts"? In general, HttpClient is some "resource" and you can just create it when you want to use it. And also it's no need to set it as "Singleton". Just wrap it in the using block. (Maybe you want to make the API wrapper as Singleton?)
public class APICaller
{
//make the APICaller singleton in some way here
//...
// the api calling method:
public string CallAPI(string someParameter)
{
var response = "";
using (var client = new HttpClient())
{
//calling the API
}
return response;
}
}
The main issue is incorrect asynchronous code.
You are using Task.Wait() which alongside asynchronous MyApiClient.GetSomeData() causes a deadlock on ASP.NET request context. That is a very common issue, see An async/await example that causes a deadlock on StackOverflow. Code with Task.Result property call is working because HttpClient.GetStringAsync() probably takes preventative measures against deadlocks. See Task.ConfigureAwait() page on MSDN and Best practice to call ConfigureAwait for all server-side code discussion on StackOverflow.
There are multiple options to write a singleton using C#. See Implementing the Singleton Pattern in C# article by Jon Skeet for a detailed overview.
As you mentioned, you can just use a static class member on the controller. HttpClient only needs to be setup once; so do this in the static constructor of the controller. Also, make sure that you use async/await for async methods, especially with long running http requests. IOC and an abstraction layer would make sense depending on your needs.
using System;
using System.Net.Http;
using System.Threading.Tasks;
namespace TestApi
{
public class MyController : Controller
{
private const string ApiUrlString = "https://apiUrl.com";
private static readonly Uri ApiUri = new Uri(ApiUrlString);
private static readonly HttpClient RestClient;
static MyController()
{
this.RestClient = new HttpClient{
BaseAddress = ApiUri
}
this.RestClient.DefaultRequestHeaders.Accept.Clear();
this.RestClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
RestClient.DefaultRequestHeaders.TryAddWithoutValidation("APIAccessToken", "token1");
RestClient.DefaultRequestHeaders.TryAddWithoutValidation("UserToken", "token2");
}
public async Task<IActionResult> ApiTest()
{
return this.Ok(await this.RestClient.GetStringAsync("somedata/search?text=test"));
}
}
}
Related
I'm working on creating unit tests using moq and I'm having trouble figuring out how to apply this framework to a portion of my program that deals with using an HttpClient. There's various resources I found that demonstrate how to mock an HttpClient response directly but the way my application makes use of HttpClient is slightly different with the utilization of Threads.
The test's skeleton:
public class MyTestClass
{
public void myTest()
{
ClassA classObj = new ClassA();
classObj.Start();
// I'd like to use moq somewhere here to mock the response that occurs in DoThreadStuff() below
}
}
The class under testing:
public class ClassA
{
private readonly Thread _myThread;
private HttpClient _client;
public ClassA()
{
// initialize some values
_myThread = new Thread(DoThreadStuff);
}
public void Start()
{
_myThread.Start(); // starts DoThreadStuff()
}
private void DoThreadStuff()
{
var newClient = getNewHttpClient(); // utility function returns a HttpClient
var response = newClient.GetAsync("/my/api/status/endpoint");
}
}
As you can see, when ClassA.Start() gets called, a new HttpClient gets created and used via GetAsync. What would the correct way to structure a test for this look like? Will I have to change the implementation of my existing classes to accommodate for Moq? Does anyone have experience with something very similar which I could take a look at?
Let's suppose that your ClassA looks like this:
public class ClassA
{
private readonly Thread _myThread;
public ClassA()
{
_myThread = new Thread(DoThreadStuff);
}
public void Start()
{
_myThread.Start();
}
private void DoThreadStuff()
{
var newClient = getNewHttpClient();
var response = newClient.GetAsync("https://httpstat.us//200").GetAwaiter().GetResult();
if(response.StatusCode == HttpStatusCode.OK)
Console.WriteLine("OK");
else if(response.StatusCode == HttpStatusCode.InternalServerError)
Console.WriteLine("Not good");
}
protected virtual HttpClient getNewHttpClient()
{
return new HttpClient();
}
}
For the sake of testability I've added some dummy code after the getNewHttpClient call
Please note that calling an async method in sync fashion (.GetAwaiter().GetResult()) is not really a good idea
I've also added the getNewHttpClient to your class as protected virtual to be able to overwrite it easily
Now let's create a helper method to be able to mock an HttpClient:
public static HttpClient SetupMockClient(HttpResponseMessage response)
{
var mockMessageHandler = new Mock<HttpMessageHandler>();
mockMessageHandler.Protected()
.Setup<Task<HttpResponseMessage>>("SendAsync", ItExpr.IsAny<HttpRequestMessage>(), ItExpr.IsAny<CancellationToken>())
.ReturnsAsync(response);
return new HttpClient(mockMessageHandler.Object);
}
Let's derive from the ClassA to override the getNewHttpClient method
internal class VerifiableClassA : ClassA
{
private HttpClient mockedHttpClient;
public VerifiableClassA(HttpClient mockedHttpClient)
{
this.mockedHttpClient = mockedHttpClient;
}
protected override HttpClient getNewHttpClient()
{
return this.mockedHttpClient;
}
}
Please note that you can do this with moq as well so you don't need to introduce a new class just for testing. But in that case the getNewHttpClient should be public.
Now you can perform unit testing like this:
var mockedClient = SetupMockClient(new HttpResponseMessage
{
StatusCode = HttpStatusCode.InternalServerError,
Content = new StringContent("Failure")
});
var sut = new VerifiableClassA(mockedClient);
sut.Start();
You should consider to use a Factory/Repository-Pattern. They make it easier to Test your methods because you can just inject mocks of objects like HttpClient. The answer Peter Csala suggested is pretty dirty because you create a class that has the single purpose to be used in a test.
With HttpClientFactory we can configure dependency injection to create and manage the lifetime of HttpClients, but this creates a lot of code understanding and transparency problems:
public class GitHubService
{
private readonly HttpClient _client;
private readonly string _repositoryName;
public GitHubService(HttpClient client, string repositoryName)
{
_client = client;
_repositoryName = repositoryName;
}
public async Task<IEnumerable<GitHubIssue>> GetAspNetDocsIssues()
{
var response = await _client.GetAsync(
$"/repos/aspnet/{_repositoryName}/issues?state=open&sort=created&direction=desc");
response.EnsureSuccessStatusCode();
using var responseStream = await response.Content.ReadAsStreamAsync();
return await JsonSerializer.DeserializeAsync
<IEnumerable<GitHubIssue>>(responseStream);
}
}
Then in Startup.cs we configure DI:
services.AddHttpClient<GitHubService>(c =>
{
c.BaseAddress = new Uri("https://api.github.com/");
// Github API versioning
c.DefaultRequestHeaders.Add("Accept", "application/vnd.github.v3+json");
// Github requires a user-agent
c.DefaultRequestHeaders.Add("User-Agent", "HttpClientFactory-Sample");
});
However, this raises a number of problems:
If I share the GitHubService code with someone, they will not understand that the class is using a non-standard HttpClient. To understand the logic of building requests, headers, authorization, you need to additionally study the Startup.cs class.
You might think that we are setting up an implementation of only the HttpClient for the GitHubService class. However, in fact, we are setting up dependency injection for the entire GitHubService class, without the possibility of managing its lifecycle and creation factory.
If the GitHubService class requires additional parameters in the constructor, we cannot configure them, because we do not have access to control the creation of the object.
Why not just inject a typed HttpClient<GitHubService> (like it does with ILogger<T>) that will not affect the main class and make it clear that you are not using a regular HttpClient? How can this problem be solved?
public class GitHubService
{
private readonly HttpClient<GitHubService> _client;
private readonly string _repositoryName;
public GitHubService(HttpClient<GitHubService> client, string repositoryName)
{
_client = client;
_repositoryName = repositoryName;
}
// Code removed for brevity.
}
I would like to call a third party API which provided us two different authorization token values. So we are getting two different sets of results back via invoking the same endpoint URL.
EndpointUrl: https://mail.yahoo.com/
Authorization: Token User123
//Do something with the response for User123
Authorization: Token User345
//Do something with the response for User345
In my client service, my wrapper function should invoke this API by calling it twice with different token values. Get the result and merge it.
Here is my service.
public class MailService : IMailService
{
private readonly HttpClient _httpClient;
public MailService(HttpClient httpClient)
{
_httpClient = httpClient;
}
public async Task<UserResponse> GetUserResponse()
{
var uri = new Uri(_httpClient.BaseAddress.AbsoluteUri + "/user-data/");
var response = await _httpClient.GetAsync(uri);
return response;
}
}
I was using Typed Client:
services.AddHttpClient<IMailService,MailService>(client =>
{
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Token", "User123");
client.BaseAddress = new Uri("https://mail.yahoo.com/");
})
Problem Statement:
What is the best way to retrieve the results? I am not sure if I should create two separate instances of HttpClient? I would like to avoid repetitive code if I go with two different classes containing their own HttpClient.
Maybe my solution lies somewhere in Named Client. I just don't know how to implement that gracefully.
Any help would be appreciated.
You can delegate adding the token header later for each message. Remove the auth header from Startup and add Http message handler (Create a new class "AuthHandler").
builder.Services.AddScoped<AuthHandler>();
services.AddHttpClient<IMailService,MailService>(client => {
client.BaseAddress = new Uri("https://mail.yahoo.com/");
})
.AddHttpMessageHandler<AuthHandler>();
In the AuthHandler, you can add the logic to retrieve and set the auth header.
The override SendAsync method will be called every time a http call is made. Below is a sample code, you can modify as per your logic/requirements:
public class AuthHandler : DelegatingHandler
{
private readonly AppSettings _appSettings;
private static string? _accessToken;
public AuthHandler(IOptions<AppSettings> options)
{
_appSettings = options.Value;
}
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request, CancellationToken cancellationToken)
{
var token = GetToken();
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);
return base.SendAsync(request, cancellationToken);
}
private string GetToken()
{
if (_accessToken != null)
{
return _accessToken;
}
_accessToken = "123";// Get your token
return _accessToken;
}
}
What is the best way to retrieve the results?
Named clients are one solution, if you are certain you will always have two authentication headers. That seems very odd to me, though.
SendAsync is another solution. By calling SendAsync instead of GetAsync, you can provide an HttpRequestMessage that can have custom headers. GetAsync is essentially a wrapper around SendAsync for convenience, and you can use the lower-level SendAsync since you need more customization.
I am going to answer, and this is more-or-less opinion based. I like my DI services to be decoupled from everything else in my project. Basically you are putting a configuration to another service in your startup.cs. I like to keep all that stuff in the service that consumes the HttpClient.
So when I inject the IHttpClientFactory, I do it by simply calling:
services.AddHttpClient();
And move on.
Now, in your MailService, you would inject it as so:
public class MailService : IMailService
{
// or store this in your applications external configuration
const Uri BaseUri = new Uri("https://mail.yahoo.com/");
const string UserDataPath = "/user-data/";
private readonly IHttpClientFactory _httpClientFactory;
public MailService(IHttpClientFactory httpClientFactory)
{
_httpClientFactory = httpClientFactory;
}
public async Task<UserResponse> GetUserResponse(string token)
{
var client = _httpClientFactory.CreateClient();
var uri = new UriBuilder(BaseUri){ Path = UserDataPath }.Uri;
using (var msg = new HttpRequestMessage(HttpMethod.Get, uri))
{
msg.Headers.Authorization = new AuthenticationHeaderValue("Token", token);
using (var resp = await _client.SendAsync(msg).ConfigureAwait(false))
{
resp.EnsureSuccessStatusCode();
return await resp.Content.ReadAsAsync<UserResponse>().ConfigureAwait(false);
}
}
}
}
Keep in mind that you should use the HttpRequestMessage pattern and not modify any Default... properties of an HttpClient. It could be in use some where else in your application at that time.
Using HttpRequestMessage ensures you get a fresh set of headers that no one else can modify. To sum it up: HttpClient.Headers are not thread-safe.
ETA:
Now that I think about it... that really is your question. You want to use two different headers using one HttpClient. It won't work in the scenario you presented. You would have to have two HttpClient with their own distinct headers.
So, if you don't want to do it the old-school way I presented, you should consider "Named" clients... each configuration has a different name.
We are currently using the HttpClient to invoke the Web APIs from the MVC application.
The HttpClient is part of a static helper class as shown below
public static class ApiClient
{
private static HttpClient MyHttpClient()
{
HttpClient client = new HttpClient();
...
return client;
}
public static T HttpGet<T>(string requestUri)
{
using (var client = MyHttpClient())
{
...
}
}
}
and it is invoked from the MVC controller as given below
ApiClient.HttpGet<MyModel>("<<API URL>>");
So whenever the ApiClient is invoked, a new underlying connection will be opened which isn't the right way.
I read about HttpClientFactory and read this post and I resulted in modifying the creation logic as
private static HttpClient MyHttpClient()
{
var serviceProvider = new ServiceCollection().AddHttpClient().BuildServiceProvider();
var httpClientFactory = serviceProvider.GetService<IHttpClientFactory>();
var client = httpClientFactory.CreateClient();
//HttpClient client = new HttpClient();
...
return client;
}
Will this prevent from opening multiple connections even if invoked multiple times?
The IHttpClientFactory functionality is predicated on dependency injection, and statics are fundamentally incompatible with dependency injection. As the docs clearly show, the correct way to do this is:
public class ApiClient
{
private readonly HttpClient _client;
public ApiClient(HttpClient client)
{
_client = client;
}
...
}
And then you register this service in ConfigureServices:
services.AddHttpClient<ApiClient>(c => { ... });
The client class should not be static and there's no reason for it to be static.
I just realized that we should reuse HttpClient other than dispose them again and again. However, as my app is on asp.net, if I don't do this:
using (var client = new HttpClient())
{
client.SendAsync(someurl, something);
}
Instead, put client in my controller as a static variable for example:
public class MyController{
public static HttpClient MyClient;
public class MyController()
{
////some configuration for the HttpClient
}
public async Task<Model> get()
{
await MyClient.PostAsJsonAsync(url, someModel);
return something
}
}
When multi-users call my api, will it trigger thread related issues?
This msdn article does say PostAsync is thread safe. https://msdn.microsoft.com/en-us/library/system.net.http.httpclient(v=vs.110).aspx#Anchor_5
Will it apply to my usage scenario? Also if there are hundred of concurrent requests, will single HttpClient cope with that?