We Keep Coding

Using Moq to mock HttpClient in the context of Thread

I'm working on creating unit tests using moq and I'm having trouble figuring out how to apply this framework to a portion of my program that deals with using an HttpClient. There's various resources I found that demonstrate how to mock an HttpClient response directly but the way my application makes use of HttpClient is slightly different with the utilization of Threads.
The test's skeleton:
public class MyTestClass
{
public void myTest()
{
ClassA classObj = new ClassA();
classObj.Start();
// I'd like to use moq somewhere here to mock the response that occurs in DoThreadStuff() below
}
}
The class under testing:
public class ClassA
{
private readonly Thread _myThread;
private HttpClient _client;
public ClassA()
{
// initialize some values
_myThread = new Thread(DoThreadStuff);
}
public void Start()
{
_myThread.Start(); // starts DoThreadStuff()
}
private void DoThreadStuff()
{
var newClient = getNewHttpClient(); // utility function returns a HttpClient
var response = newClient.GetAsync("/my/api/status/endpoint");
}
}
As you can see, when ClassA.Start() gets called, a new HttpClient gets created and used via GetAsync. What would the correct way to structure a test for this look like? Will I have to change the implementation of my existing classes to accommodate for Moq? Does anyone have experience with something very similar which I could take a look at?

Let's suppose that your ClassA looks like this:
public class ClassA
{
private readonly Thread _myThread;
public ClassA()
{
_myThread = new Thread(DoThreadStuff);
}
public void Start()
{
_myThread.Start();
}
private void DoThreadStuff()
{
var newClient = getNewHttpClient();
var response = newClient.GetAsync("https://httpstat.us//200").GetAwaiter().GetResult();
if(response.StatusCode == HttpStatusCode.OK)
Console.WriteLine("OK");
else if(response.StatusCode == HttpStatusCode.InternalServerError)
Console.WriteLine("Not good");
}
protected virtual HttpClient getNewHttpClient()
{
return new HttpClient();
}
}
For the sake of testability I've added some dummy code after the getNewHttpClient call
Please note that calling an async method in sync fashion (.GetAwaiter().GetResult()) is not really a good idea
I've also added the getNewHttpClient to your class as protected virtual to be able to overwrite it easily
Now let's create a helper method to be able to mock an HttpClient:
public static HttpClient SetupMockClient(HttpResponseMessage response)
{
var mockMessageHandler = new Mock<HttpMessageHandler>();
mockMessageHandler.Protected()
.Setup<Task<HttpResponseMessage>>("SendAsync", ItExpr.IsAny<HttpRequestMessage>(), ItExpr.IsAny<CancellationToken>())
.ReturnsAsync(response);
return new HttpClient(mockMessageHandler.Object);
}
Let's derive from the ClassA to override the getNewHttpClient method
internal class VerifiableClassA : ClassA
{
private HttpClient mockedHttpClient;
public VerifiableClassA(HttpClient mockedHttpClient)
{
this.mockedHttpClient = mockedHttpClient;
}
protected override HttpClient getNewHttpClient()
{
return this.mockedHttpClient;
}
}
Please note that you can do this with moq as well so you don't need to introduce a new class just for testing. But in that case the getNewHttpClient should be public.
Now you can perform unit testing like this:
var mockedClient = SetupMockClient(new HttpResponseMessage
{
StatusCode = HttpStatusCode.InternalServerError,
Content = new StringContent("Failure")
});
var sut = new VerifiableClassA(mockedClient);
sut.Start();

You should consider to use a Factory/Repository-Pattern. They make it easier to Test your methods because you can just inject mocks of objects like HttpClient. The answer Peter Csala suggested is pretty dirty because you create a class that has the single purpose to be used in a test.

Multiple httpclient instances with same implementation in dotnet core

I would like to call a third party API which provided us two different authorization token values. So we are getting two different sets of results back via invoking the same endpoint URL.
EndpointUrl: https://mail.yahoo.com/
Authorization: Token User123
//Do something with the response for User123
Authorization: Token User345
//Do something with the response for User345
In my client service, my wrapper function should invoke this API by calling it twice with different token values. Get the result and merge it.
Here is my service.
public class MailService : IMailService
{
private readonly HttpClient _httpClient;
public MailService(HttpClient httpClient)
{
_httpClient = httpClient;
}
public async Task<UserResponse> GetUserResponse()
{
var uri = new Uri(_httpClient.BaseAddress.AbsoluteUri + "/user-data/");
var response = await _httpClient.GetAsync(uri);
return response;
}
}
I was using Typed Client:
services.AddHttpClient<IMailService,MailService>(client =>
{
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Token", "User123");
client.BaseAddress = new Uri("https://mail.yahoo.com/");
})
Problem Statement:
What is the best way to retrieve the results? I am not sure if I should create two separate instances of HttpClient? I would like to avoid repetitive code if I go with two different classes containing their own HttpClient.
Maybe my solution lies somewhere in Named Client. I just don't know how to implement that gracefully.
Any help would be appreciated.

You can delegate adding the token header later for each message. Remove the auth header from Startup and add Http message handler (Create a new class "AuthHandler").
builder.Services.AddScoped<AuthHandler>();
services.AddHttpClient<IMailService,MailService>(client => {
client.BaseAddress = new Uri("https://mail.yahoo.com/");
})
.AddHttpMessageHandler<AuthHandler>();
In the AuthHandler, you can add the logic to retrieve and set the auth header.
The override SendAsync method will be called every time a http call is made. Below is a sample code, you can modify as per your logic/requirements:
public class AuthHandler : DelegatingHandler
{
private readonly AppSettings _appSettings;
private static string? _accessToken;
public AuthHandler(IOptions<AppSettings> options)
{
_appSettings = options.Value;
}
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request, CancellationToken cancellationToken)
{
var token = GetToken();
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);
return base.SendAsync(request, cancellationToken);
}
private string GetToken()
{
if (_accessToken != null)
{
return _accessToken;
}
_accessToken = "123";// Get your token
return _accessToken;
}
}

What is the best way to retrieve the results?
Named clients are one solution, if you are certain you will always have two authentication headers. That seems very odd to me, though.
SendAsync is another solution. By calling SendAsync instead of GetAsync, you can provide an HttpRequestMessage that can have custom headers. GetAsync is essentially a wrapper around SendAsync for convenience, and you can use the lower-level SendAsync since you need more customization.

I am going to answer, and this is more-or-less opinion based. I like my DI services to be decoupled from everything else in my project. Basically you are putting a configuration to another service in your startup.cs. I like to keep all that stuff in the service that consumes the HttpClient.
So when I inject the IHttpClientFactory, I do it by simply calling:
services.AddHttpClient();
And move on.
Now, in your MailService, you would inject it as so:
public class MailService : IMailService
{
// or store this in your applications external configuration
const Uri BaseUri = new Uri("https://mail.yahoo.com/");
const string UserDataPath = "/user-data/";
private readonly IHttpClientFactory _httpClientFactory;
public MailService(IHttpClientFactory httpClientFactory)
{
_httpClientFactory = httpClientFactory;
}
public async Task<UserResponse> GetUserResponse(string token)
{
var client = _httpClientFactory.CreateClient();
var uri = new UriBuilder(BaseUri){ Path = UserDataPath }.Uri;
using (var msg = new HttpRequestMessage(HttpMethod.Get, uri))
{
msg.Headers.Authorization = new AuthenticationHeaderValue("Token", token);
using (var resp = await _client.SendAsync(msg).ConfigureAwait(false))
{
resp.EnsureSuccessStatusCode();
return await resp.Content.ReadAsAsync<UserResponse>().ConfigureAwait(false);
}
}
}
}
Keep in mind that you should use the HttpRequestMessage pattern and not modify any Default... properties of an HttpClient. It could be in use some where else in your application at that time.
Using HttpRequestMessage ensures you get a fresh set of headers that no one else can modify. To sum it up: HttpClient.Headers are not thread-safe.
ETA:
Now that I think about it... that really is your question. You want to use two different headers using one HttpClient. It won't work in the scenario you presented. You would have to have two HttpClient with their own distinct headers.
So, if you don't want to do it the old-school way I presented, you should consider "Named" clients... each configuration has a different name.

Flurl Testing: How to setup response based on conditions?

I have a method that makes two HTTP calls to two different urls using Flurl. I need to unit test this method in which I want Flurl to respond with two different responses. How can I set this up?
My test class is like:
public class SUT
{
public async Task Mut(object obj)
{
var x = await url1.PostJsonAsync(obj).ReceiveJson();
if ((bool)x.property = true)
{
var y = await url2.GetJsonAsync();
// Process y.
}
}
}
I have a test class as below:
public class TestSut : Disposable
{
private readonly HttpTest httpTest;
public TestSut()
{
httpTest = new HttpTest();
}
[Fact]
public async Task TestMut()
{
// call Mut...
}
public void Dispose()
{
httpTest?.Dispose();
}
}
What I would like is something along the lines of:
httpTest.ForUrl(url1).ResponsdWithJson(...);
httpTest.ForUrl(url2).ResponsdWithJson(...);

The short answer is no, you can't configure different behavior by URL today, but it's coming in 3.0. I'll update this when it's released (or hopefully someone else will if I forget :).
In this particular case though, assuming your SUT code at least somewhat resembles the real code you're targeting, it looks like url1 will always be called before url2, so if you just queue the responses in the same order, Flurl will guarantee that they are returned in the same order.
httpTest
.ResponsdWithJson(/* fake response for url1 */)
.ResponsdWithJson(/* fake response for url2 */);
Of course the SUT may not actually call things in a determinate order like that, in which case you'll have to wait for 3.0 unfortunately.

Setting member in HttpClientHandler results in "This instance has already started one or more requests" error

Here is my code:
public class Uploader
{
private readonly HttpMessageHandler m_httpMessageHandler;
public Uploader(HttpMessageHandler httpMessageHandler)
{
m_httpMessageHandler = httpMessageHandler;
}
public async Task<string> Upload(string url, Dictionary<string, string> data)
{
HttpResponseMessage result;
try
{
if (somecheck)
{
HttpClientHandler handler = (HttpClientHandler)m_httpMessageHandler;
// error occurs here
handler.ServerCertificateCustomValidationCallback = delegate { return true; };
}
var client = new HttpClient(m_httpMessageHandler);
result = await client.PostAsync(url, new FormUrlEncodedContent(data));
if (result.StatusCode != HttpStatusCode.OK)
{
return Strings.LabelPrinterNotConfiguredError;
}
else
{
return null; // Success!
}
}
catch(Exception e)
{
// do some stuff
}
}
}
In my Startup.cs
services.AddSingleton<HttpMessageHandler, HttpClientHandler>();
The reason why I have this is because I've created a mock HttpClientHandler used for unit testing the HttpClient. This way I can extend from the abstract class HttpMessageHandler for the mock implementation in my tests.
I am getting this error:
System.InvalidOperationException:
This instance has already started one or more requests. Properties can
only be modified before sending the first request.
After reading through several other answers on SO, I know that the issue is that the single HttpClientHandler that I have is somehow having its properties modified after starting which the runtime does not like. However, I still can't figure out why that is happening because it does not appear that I am overtly modifying BaseAddress, Timeout, or MaxResponseContentBufferSize.
The answers appear to be to modify the HttpClientHandler after creating it but if so, how do I set the ServerCertificateCustomValidationCallback member variable appropriately?
The follow-up question would be: how would I fix this error while also keeping the ability to mock/test HttpClient?

Creating HttpClient in older ASP.NET MVC app without Startup

I have an old version of ASP.NET MVC app that doesn't have a Startup.cs. I wanted to implement a clean way to have an HttpClient that I would use for my API calls to third parties.
Here's what I've done so far based on some ideas/recommendations I've received for this question. The problem is that when I make the API call, it goes nowhere. I put it in a try catch but I'm not even getting an exception. The API provider tells me that they're not seeing the search parameter.
First, I created this HttpClientAccessor for lazy loading.
public static class HttpClientAccessor
{
public static Func<HttpClient> ValueFactory = () =>
{
var client = new HttpClient();
client.BaseAddress = new Uri("https://apiUrl.com");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.TryAddWithoutValidation("APIAccessToken", "token1");
client.DefaultRequestHeaders.TryAddWithoutValidation("UserToken", "token2");
return client;
};
private static Lazy<HttpClient> client = new Lazy<HttpClient>(ValueFactory);
public static HttpClient HttpClient
{
get { return client.Value; }
}
}
I then created an API client of my own so that I can have the API call functions in one place which looks like this:
public class MyApiClient
{
public async Task GetSomeData()
{
var client = HttpClientAccessor.HttpClient;
try
{
var result = await client.GetStringAsync("somedata/search?text=test");
var output = JObject.Parse(result);
}
catch(Exception e)
{
var error = e.Message;
}
}
}
Then in my ASP.NET Controller action, I do this:
public class MyController : Controller
{
private static readonly MyApiClient _apiClient = new MyApiClient ();
public ActionResult ApiTest()
{
var data = _apiClient.GetSomeData().Wait();
}
}
Any idea where my mistake is?
UPDATE:
This simple approach works fine:
public class MyController : Controller
{
private static readonly HttpClient _client = new HttpClient();
public ActionResult ApiTest()
{
_client.BaseAddress = new Uri("https://apiUrl.com");
_client.DefaultRequestHeaders.Accept.Clear();
_client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
_client.DefaultRequestHeaders.TryAddWithoutValidation("APIAccessToken", "token1");
_client.DefaultRequestHeaders.TryAddWithoutValidation("UserToken", "token2");
var response = _client.GetStringAsync("somedata/search?text=test").Result;
}
}

As mentioned, dependency injection is not being utilized so technically there is no need for a composition root where these things would have been initialized.
If there is no need to actually initialize the client on start up you could consider using a Lazy singleton approach.
An example
public static class HttpClientAccessor {
public static Func<HttpClient> ValueFactory = () => {
var client = new HttpClient();
client.BaseAddress = new Uri("https://apiUrl.com");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.TryAddWithoutValidation("APIAccessToken", "token1");
client.DefaultRequestHeaders.TryAddWithoutValidation("UserToken", "token2");
return client;
};
private static Lazy<HttpClient> client = new Lazy<HttpClient>(ValueFactory);
public static HttpClient HttpClient {
get { return client.Value; }
}
}
The factory delegate of the Lazy<HttpClient> can be made more complex if additional settings are needed on the client.
And where ever the client is needed you call the service
var client = HttpClientAccessor.HttpClient;
var response = await client.GetStringAsync("{url}");
the client will be initialized on first use and you will get the same instance on subsequent calls for the instance.
As used in your controller, you are mixing async calls with blocking calls line .Wait() or .Result. This can lead to deadlocks and should be avoided.
public class MyController : Controller {
private static readonly MyApiClient _apiClient = new MyApiClient ();
public async Task<ActionResult> ApiTest() {
var data = await _apiClient.GetSomeData();
//...
}
}
Code should be async all the way through.
Reference Async/Await - Best Practices in Asynchronous Programming

The Application_Start() method is the right place. But I would have to ask: why you have to create the HttpClient instance when the "application starts"? In general, HttpClient is some "resource" and you can just create it when you want to use it. And also it's no need to set it as "Singleton". Just wrap it in the using block. (Maybe you want to make the API wrapper as Singleton?)
public class APICaller
{
//make the APICaller singleton in some way here
//...
// the api calling method:
public string CallAPI(string someParameter)
{
var response = "";
using (var client = new HttpClient())
{
//calling the API
}
return response;
}
}

The main issue is incorrect asynchronous code.
You are using Task.Wait() which alongside asynchronous MyApiClient.GetSomeData() causes a deadlock on ASP.NET request context. That is a very common issue, see An async/await example that causes a deadlock on StackOverflow. Code with Task.Result property call is working because HttpClient.GetStringAsync() probably takes preventative measures against deadlocks. See Task.ConfigureAwait() page on MSDN and Best practice to call ConfigureAwait for all server-side code discussion on StackOverflow.
There are multiple options to write a singleton using C#. See Implementing the Singleton Pattern in C# article by Jon Skeet for a detailed overview.

As you mentioned, you can just use a static class member on the controller. HttpClient only needs to be setup once; so do this in the static constructor of the controller. Also, make sure that you use async/await for async methods, especially with long running http requests. IOC and an abstraction layer would make sense depending on your needs.
using System;
using System.Net.Http;
using System.Threading.Tasks;
namespace TestApi
{
public class MyController : Controller
{
private const string ApiUrlString = "https://apiUrl.com";
private static readonly Uri ApiUri = new Uri(ApiUrlString);
private static readonly HttpClient RestClient;
static MyController()
{
this.RestClient = new HttpClient{
BaseAddress = ApiUri
}
this.RestClient.DefaultRequestHeaders.Accept.Clear();
this.RestClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
RestClient.DefaultRequestHeaders.TryAddWithoutValidation("APIAccessToken", "token1");
RestClient.DefaultRequestHeaders.TryAddWithoutValidation("UserToken", "token2");
}
public async Task<IActionResult> ApiTest()
{
return this.Ok(await this.RestClient.GetStringAsync("somedata/search?text=test"));
}
}
}