We Keep Coding

Post HTTP request without awaiting the result

I have the following endpoint:
[HttpPost("Submit")]
public String post()
{
_ = _service.SubmitMetric("test", MetricType.Count, 60, 1);
return "done";
}
And the service implementation:
public Task<HttpResponseMessage> SubmitMetric(<params>)
{
// build payload
using (var httpClient = new HttpClient())
{
return httpClient.PostAsync(<params>);
}
}
When I run the code and call the endpoint, the HTTP POST is not triggered. However, if I change my code to:
public async Task<HttpResponseMessage> SubmitMetric(<params>)
{
// build payload
using (var httpClient = new HttpClient())
{
return await httpClient.PostAsync(<params>);
}
}
the POST is submitted as expected. Why is that happening, and what can I do if I don't really care about the HTTP response? I just want to submit it and continue my flow. Shouldn't I be able to use it without awaiting the result? For example:
public void SubmitMetric(<params>)
{
// build payload
using (var httpClient = new HttpClient())
{
httpClient.PostAsync(<params>);
}
}

There are two problems with this code. If either was fixed, there would be no problem:
The HttpClient is used incorrectly. An HttpClient object is thread-safe and meant to be reused, not disposed. Disposing it like this leaks sockets and can result in application crashes or worse, instability. An HttpClient resolves the URL's Host to a socket and caches that socket. The OS also caches opened sockets because opening them is expensive. They're kept alive for a while even if an application closes them because some packets may still be in transit
By not awaiting PostAsync execution exits the using block and the HttpClient instance is disposed before the request had a chance to even start.
In any case, making a POST doesn't take long so there's no need to make the method fire-and-forget. Besides, few applications are OK with losing metrics, especially when things go wrong. That's when metrics are most useful.
Which is why ASP.NET Core 6 adds built-in support for OpenTelemetry tracing and metrics. More on that at the end, but the supporting packages can be used in ASP.NET Framework as well. You may be able to replace your current service with a built-in one.
Use await - not enough
One way to fix this is to use await but that doesn't solve the HttpClient usage problem.
public async Task<HttpResponseMessage> SubmitMetric(<params>)
{
// build payload
using (var httpClient = new HttpClient())
{
return await httpClient.PostAsync(<params>);
}
}
At the very least the HttpClient should be stored in a field. Once that's done though, there's no longer any reason to await, provided the service itself is still around :
HttpClient httpClient = new HttpClient();
public Task<HttpResponseMessage> SubmitMetric(<params>)
{
return httpClient.PostAsync(<params>);
}
Long lived services
Which brings us to keeping the service around. In ASP.NET and ASP.NET Core each request is served by a separate thread, in a new instance of the Controller class. The request itself is used as a GC scope so anything created during a request is disposed once this concludes, including the HttpClient instance.
To keep the Metrics service around we need to either register it as Singleton in ASP.NET Core's DI, make it a BackgroundService or ensure it's a singleton in ASP.NET Framework. We could make the field static, but that leads to the next issue.
Proper HttpClient usage
HttpClient can still cause problems if used as a singleton. The HttpClient caches sockets to specific machines. If that machine goes away, the HttpClient will still try to communicate with it causing errors. This can happen easily when the remote services uses a load balancer or fails over to a new server. To fix this, the HttpClient instance or rather the sockets, need to be recycled periodically.
That's the job of the HttpClientFactory. This class caches and recycles SocketClientHandler instances, the classes that do the actual work in an HttpClient. These are recycled periodically, eg every 10 minutes. When asked for a new HttpClient instance, it creates a new instance wrapping one of the already available handlers.
When you use services.AddHttpClient in ASP.NET Core you're actually configuring an HttpClientFactory. When you add an HttpClient dependency in a controller, the instance will be created by the configured HttpClientFactory.
This means that the following action would work properly :
HttpClient _client;
public MyController(HttpClient client)
{
_client=client;
}
[HttpPost("Submit")]
public String post()
{
await _client.PostAsync(<params>);
return "done";
}
A scoped service with an HttpClient dependency would also work:
MyService _service;
public MyController(MyService service)
{
_service=service;
}
HttpPost("Submit")]
public String post()
{
await _service.SubmitMetric("test", MetricType.Count, 60, 1);
return "done";
}
where MyService is :
class MyService
{
HttpClient _client;
public MyService(HttpClient client)
{
_client=client;
}
public Task<HttpResponseMessage> SubmitMetric(<params>)
{
// build payload
return httpClient.PostAsync(<params>);
}
}
In this case there's no real need to await inside SubmitMetric, that's taken care of by the action.
Using the built-in OpenTelemetry tracing and metrics
ASP.NET Core 6, the upcoming Long-Term-Support version, adds native support for the OpenTelemetry standard for logging, tracing and metrics. This allows using a standard API to push metrics to a lot of different observability applications like Prometheus, Jaeger, Zipking, Elastic and Splunk.
Instead of rolling one's own metrics infrastructure it's better to use the standard API. OpenTelemetry for .NET supports this in ASP.NET Framework 4.6 and later. ASP.NET Core 5 and later are instrumented to publish metrics and tracing to OpenTelemetry providers through the built-in System.Diagnostics namespace and the Activity class.
In fact, Controller is already instrumented so you could get rid of the metrics service, adding any Tags and Baggage to the request's current activity:
[HttpPost("Submit")]
public String post()
{
Activity.Current?.AddTag("test");
...
return "done";
}
Metrics were added in ASP.NET Core 6 Preview 5:
Meter meter = new Meter("my.library.meter.name", "v1.0");
Counter<int> _counter;
public MyController(...)
{
_counter = meter.CreateCounter<int>("Requests");
}
[HttpPost("Submit")]
public String post()
{
counter.Add(60, KeyValuePair.Create<string, object>("request", "test"));
return "done";
}

Don't do it. Await for it even though you discard the result.
Fire and forget is an anti pattern and the context that you are performing the request can be invalidated/killed before the request could be completed, terminating the connection. Just await it, and don't do anything with the result.

httpClient will be disposed while the POST operation is running, probably resulting in killing the socket. If you use await, the object will remain inside the using clause while the operation is running, and it won't be terminated before it finishes.
Note that in your current implementation, you're creating a new connection on each API request, which might eventually lead to socket exhaustion. A better approach would be injecting IHttpClientFactory, which manage the lifetime of network connections for you, and reuses connections from the pool:
public class MyService
{
private readonly IHttpClientFactory _httpClient;
public MyService(IHttpClientFactory httpClient)
{
_httpClient = httpClient;
}
public async Task<HttpResponseMessage> SubmitMetric(/*<params>*/)
{
var httpClient = _httpClient.CreateClient();
return await httpClient.PostAsync(/*<params>*/);
}
}
Note: You need to add services.AddHttpClient() in ConfigureServices in your Startup.cs to enable injection.

How do I implement IHttpClientFactory with DI for unittesting purposes? [duplicate]

I am using the new Web API bits in a project, and I have found that I cannot use the normal HttpMessageRequest, as I need to add client certificates to the request. As a result, I am using the HttpClient (so I can use WebRequestHandler). This all works well, except that it isn't stub/mock friendly, at least for Rhino Mocks.
I would normally create a wrapper service around HttpClient that I would use instead, but I would like to avoid this if possible, as there are a lot of methods that I would need to wrap. I am hoping that I have missing something—any suggestions on how to stub HttpClient?

As an alternative to the excellent ideas already presented by #Raj, it may be possible to go a step lower and to mock/fake the HttpMessageHandler instead.
If you make any class that needs an HttpClient accept it as a dependency injection parameter in the constructor, then when unit testing you can pass in an HttpClient that has been injected with your own HttpMessageHandler. This simple class has only one abstract method that you need to implement, as follows:
public class FakeHttpMessageHandler : HttpMessageHandler
{
public HttpRequestMessage RequestMessage { get; private set; }
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
RequestMessage = request;
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK));
}
}
My trivial example just saves the HttpRequestMessage in a public property for later inspection and returns HTTP 200 (OK), but you could augment this by adding a constructor that sets the result you want returned.
You'd use this class like this:
public void foo()
{
//Arrange
var fakeHandler = new FakeHttpMessageHandler();
var client = new HttpClient(fakeHandler);
var SUT = new ClassUnderTest(client);
//Act
SUT.DomSomething();
//Assert
fakeHandler.RequestMessage.Method.ShouldEqual(HttpMethod.Get); // etc...
}
There are limitations to this approach, for example in a method that makes multiple requests or needs to create multiple HttpClients, then the fake handler might start to become too complicated. However, it may be worth consideration for simple cases.

I released a library a few months ago called MockHttp which might be useful. It uses a custom HttpMessageHandler with a fluent (and extensible) API. You can inject the mocked handler (or HttpClient) into your service class and it will respond as it was configured.
Below shows basic usage. The When and Respond methods have a bunch of overloads, including running custom logic. The documentation on the GitHub page goes into a lot more detail.
var mockHttp = new MockHttpMessageHandler();
// Setup a respond for the user api (including a wildcard in the URL)
mockHttp.When("http://localhost/api/user/*")
.Respond("application/json", "{'name' : 'Test McGee'}"); // Respond with JSON
// Inject the handler or client into your application code
var client = new HttpClient(mockHttp);
var response = async client.GetAsync("http://localhost/api/user/1234");
// or without async: var response = client.GetAsync(...).Result;
var json = await response.Content.ReadAsStringAsync();
// No network connection required
Console.Write(json); // {'name' : 'Test McGee'}

I use Moq and I can stub out the HttpClient. I think this the same for Rhino Mock (I haven’t tried by myself).
If you just want to stub the HttpClient the below code should work:
var stubHttpClient = new Mock<HttpClient>();
ValuesController controller = new ValuesController(stubHttpClient.Object);
Please correct me if I’m wrong. I guess you are referring to here is that stubbing out members within HttpClient.
Most popular isolation/mock object frameworks won’t allow you to stub/setup on non- virtual members
For example the below code throws an exception
stubHttpClient.Setup(x => x.BaseAddress).Returns(new Uri("some_uri");
You also mentioned that you would like to avoid creating a wrapper because you would wrap lot of HttpClient members. Not clear why you need to wrap lots of methods but you can easily wrap only the methods you need.
For example :
public interface IHttpClientWrapper { Uri BaseAddress { get; } }
public class HttpClientWrapper : IHttpClientWrapper
{
readonly HttpClient client;
public HttpClientWrapper() {
client = new HttpClient();
}
public Uri BaseAddress {
get
{
return client.BaseAddress;
}
}
}
The other options that I think might benefit for you (plenty of examples out there so I won’t write the code)
Microsoft Moles Framework
http://research.microsoft.com/en-us/projects/moles/
Microsoft Fakes: (if you are using VS2012 Ultimate)
http://msdn.microsoft.com/en-us/library/hh549175.aspx

Dependency injection: HttpClient or HttpClientFactory?

Everywhere I can see three main approaches to create clients (basic, named, typed) in DI, but I have found nowhere if to inject IHttpClientFactory or HttpClient (both possible).
Q1: What is the difference between injecting IHttpClientFactory or HttpClient please?
Q2: And if IHttpClientFactory is injected, should I use factory.CreateClient() for each call?

Summary
HttpClient can only be injected inside Typed clients
for other usages, you need IHttpClientFactory
In both scenarios, the lifetime of HttpClientMessageHandler is managed by the framework, so you are not worried about (incorrectly) disposing the HttpClients.
Examples
In order to directly inject HttpClient, you need to register a specific Typed service that will receive the client:
services.AddHttpClient<GithubClient>(c => c.BaseAddress = new System.Uri("https://api.github.com"));
Now we can inject that inside the typed GithubClient
public class GithubClient
{
public GithubClient(HttpClient client)
{
// client.BaseAddress is "https://api.github.com"
}
}
You can't inject the HttpClient inside AnotherClient, because it is not typed to AnotherClient
public class AnotherClient
{
public AnotherClient(HttpClient client)
{
// InvalidOperationException, can't resolve HttpClient
}
}
You can, however:
1. Inject the IHttpClientFactory and call CreateClient(). This client will have BaseAddress set to null.
2. Or configure AnotherClient as a different typed client with, for example, a different BaseAdress.
Update
Based on your comment, you are registering a Named client. It is still resolved from the IHttpClientFactory.CreateClient() method, but you need to pass the 'name' of the client
Registration
services.AddHttpClient("githubClient", c => c.BaseAddress = new System.Uri("https://api.github.com"));
Usage
// note that we inject IHttpClientFactory
public HomeController(IHttpClientFactory factory)
{
this.defaultClient = factory.CreateClient(); // BaseAddress: null
this.namedClient = factory.CreateClient("githubClient"); // BaseAddress: "https://api.github.com"
}

Sadly I cannot comment, but only Post an answer. Therefore I suggest you should check out the following Links:
https://learn.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests
https://aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/
Regarding your Questions it more or Less boils down to this:
Q1 -> IHttpClientFactory handles the connection pools of HttpClient instances and this will help you regarding load and dispose problems as discribed in the links, if the HttpClient is used wrong.
Q2 -> yes you should use factory.create client according to microsoft docs

What is the best approach to define an HttpClient instance in a base class of a class library project?

There is an application which has 3 interfaces and whoever wants to use this app needs to implement these interfaces. I have created a class library project which has these interface implementations that I have inherited all from the same base class to be able to have a single HttpClient. Here is what I have done so far:
public class BaseProxy
{
protected static readonly HttpClient Client;
static BaseProxy()
{
Client = new HttpClient();
}
}
and I have used this Client in all derived classes to make GetAsync and PostAsync requests as follows:
public class XProxyImplementation
{
var response = Client.GetAsync(BaseUrl + "XXXApi/GetClientSettings/").Result;
response.EnsureSuccessStatusCode();
}
None of the methods in Web API are async by the way and I chose singleton solution because I don't want to use using block for each request. My question is should I go for a DI solution or is this code enough for an app which will be used internally? All suggestions for improvement are welcome.
I have read many answers regarding to using DI containers but this is just a class library with proxy implementations.
My other concern is even if I want to use DI, currently I am not able to introduce DI in my constructor classes because the other application that uses my implementations is looking for an empty constructor. When I try to pass HttpClient parameter to the constructor I get the following error:
The current type, System.Net.Http.HttpMessageHandler, is an abstract
class and cannot be constructed
The application which uses my dlls doesn't allow me to pass any parameters to constructor that uses any abstract classes. I guess this application uses Unity to make the handshake and in some way it looks for an empty constructor. Once I try to do the following changes I am getting the error:
public BaseProxy() : this(Service.HttpClient)
{
}
public XProxyImplementation(HttpClient client) : base(client)
{
}
That's why I actually prefered singleton instance to DI implementation.

DI? Yes
DI will enable testability of your proxy classes, whereas your current implementation cannot be unit-tested. It will also improve separation of concerns: remove the responsibility of controlling HttpClient lifetime from the proxy.
Typically, you would do something like this:
public abstract class BaseProxy
{
protected readonly HttpClient Client;
protected BaseProxy(HttpClient client)
{
Client = client;
}
// ... other members
}
public class XProxyImplementation : BaseProxy
{
public XProxyImplementation(HttpClient client) : base(client)
{
}
// ... other members
public Task SendRequest() // for example
{
return Client.GetAsync("....");
}
}
During the tests, you would initialize a different instance of HttpClient, injecting a test-friendly implementation of HttpMessageHandler:
// you implement TestHttpMessageHandler that aids your tests
var httpClient = new HttpClient(new TestHttpMessageHandler());
var proxyUnderTest = new XProxyImplementation(httpClient);
See this blog post for explanation of unit testing with HttpClient and HttpMessageHandler.
DI container? No
Now that we introduced dependency injection into your code, next question is, what injection mechanism should be used.
In your specific case, I would vote against coupling to any specific DI container, because you want your library to be consumed by many different applications, and you don't want to bloat their dependencies (an application might already be using a different DI container).
Moreover, since the code you posted is very simple, a full-blown DI container would be an overkill. In production code, you can just move your singleton HttpClient to a "service locator":
public static class SingletonServices
{
public static readonly HttpClient HttpClient;
static SingletonServices()
{
HttpClient = new HttpClient();
}
}
So that when you instantiate a proxy in production code, you do this:
var proxy = new XProxyImplementation(SingletonServices.HttpClient);

I would definitely go with a DI solution for this using the Microsoft.Extensions.DependencyInjection package.
https://dzone.com/articles/dependency-injection-in-net-core-console-applicati
And you should also be very aware how you use your async methods like GetAsync.
Using .Result almost never gives the desired result and you would be better off making the method async and using an await keyword like so:
var response = await Client.GetAsync(BaseUrl + "XXXApi/GetClientSettings/");
https://montemagno.com/c-sharp-developers-stop-calling-dot-result/
is a good resource for the whys and hows of this best practice

DI is the answer. If you do not want to use ID there is an HttpClientFactory that you can implement.
You can read more here
https://learn.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests

Unit testing Web Service responses

I am currently writing an API wrapper in C# for ResellerClub's REST/HTTP API, which provides responses in garden-variety JSON objects. Invocation is performed by performing HTTP POST/GET on API endpoints using the HttpClient class. JSON.Net is used for parsing the responses.
How I can unit test my API wrapper functionality for the API as most calls require a level of expected state in order to succeed. For example, I cannot test the creation of a CNAME record on a domain that I have not already registered.
I understand that tests should never rely on state which they do not arrange themselves, and I've also been told that the tests should never actually deal with any kind of persistence mechanism such as a database. So, for the above example of a CNAME record, that as part of the "Arrange" phase of the test I should register a test domain, assert it worked, then do the actual CNAME function?
Alternative, should I come up with some way of mocking the JSON responses that are returned from the Reseller Club API?
EDIT: Example of my API class (ResellerClubApi.cs)
private async Task<string> DownloadString(string uri)
{
// HttpClient object downloads the JSON response string asynchronously
}
The DownloadString() method is used by my functionality as a generic means of grabbing the response from the third party service.
public async Task<List<string>> SuggestNames(string domainName)
{
// Calls DownloadString() with the correct URI, uses Newtonsoft.JSON to parse
// string representation of JSON into object
}
Methods such as SuggestNames() above are called like this from the higher service layer
public void someServiceLayerMethod()
{
var rcApi = new ResellerClubApi();
var x = rcApi.SuggestNames("something");
// ...
}
As you can see, I am a bit stuck as to how to mock JSON responses from the likes of HttpClient when my ResellerClubApi class is the lowest possible layer of my own code prior to doing things over HTTP.
I also don't know how to start using IoC to hand the HttpClient dependency...
Thanks

I would separate the code from your ResellerClubApi class which involves downloading stuff and authorization, and everything that involves connecting to a remote service, in let's say a ResellerClubClient and have it implement a IResellerClubClient interface.
public interface IResellerClubClient {
string RequestJson(string url);
}
public class ResellerClubClient : IResellerClubClient {
// implement your methods here
}
public ResellerClubApi : IResellerClubApi {
private readonly IResellerClubClient client;
// Pass the client as dependency, either manually or using Dependency framework of your choice
public ResellerClubApi(IResellerClubClient client) {
this.client = client;
}
public List<string> SuggestNames(string domainName) {
var jsonString = this.client.RequestJson("http://example.com/domains/?name="+domainName);
// decode it and do something with it
}
}
This allows you to test your ResellerClubApi class without being depending on a concrete IResellerClubClient implementation. And the best is, you can change it (from HttpClient to socket or whatever and don't ever have to touch your ResellerClubApi.
And then set up your Unit test in framework of your choice. Some example with Moq framework:
var mockedJsonString = '{ succes: true, names: ["domainA.com", "domainA.us"] }';
// create mockup object using IResellerClubClient interface
var resellerClubClient = new Mock<IResellerClubClient>();
// Tell the mock object to return "mockedJsonString" when any parameter is passed to RequestJsonString.
// If you do more than 1 call in a test, or if that's expected to be called multiple times inside
// the method to be tested, you can setup multiple conditions and results this way too
resellerClubClient.Setup(x => x.RequestJson(It.IsAny<string>())).Returns(mockedJsonString);
var api = new ResellerClubApi(resellerClubClient.Object);
List<string> names = api.SuggestNames("domain.com");
// do your assertions here
By having abstracted the connection and data retrieving methods into hit's own class represented by an interface, you made your Api class UnitTestable and easy to mock server responses.
Of course, the ResellerClubClient can't be Unit tested of course. But it can be done in an integration test or a verification test. A UnitTest should never involve connecting to a server or a database.

Here is a way to do it by mocking the HttpMessageHandler using Moq unit test. http://geekswithblogs.net/abhi/archive/2013/11/20/unit-tests-for-httpclient-using-httpmessagehandler.aspx