We Keep Coding

Performance Issue: .NET HttpClient Response is slow

Problem:
I am using .NET Class HttpClient to make Requests to the endpoint URL.
My Code:
using (HttpClient apiClient1 = new HttpClient())
{
apiClient.GetAsync(apiUrl).Result;
}
Problem Identified:
If I use using block, I'm opening multiple connections, leads to socketExceptions.
Changed My Above Code to:
public class RequestController: ApiController
{
private static HttpClient apiClient = new HttpClient();
[HttpPost]
public dynamic GetWebApiData([FromBody] ParamData params)
{
var resultContent = apiClient.GetAsync(apiUrl).Result.Content;
return Newtonsoft.Json.JsonConvert.DeserializeObject<object>(resultContent.ReadAsStringAsync().Result);
}
}
Result of the above code after making HttpClient as static is as follows:
Only one Connection is established.
For Each request, I'm looking for the 200 milliseconds reduction in
Response Time.
What I Need:
I want to make conurrent calls atleast 50 calls to the end point with High-Speed response.
Kindly help me with this scenario.

Use the async API and stop calling .Result blocking calls.
public class RequestController: ApiController {
private static HttpClient apiClient = new HttpClient();
[HttpPost]
public async Task<IHttpActionResult> GetWebApiData([FromBody] ParamData data) {
var response = await apiClient.GetAsync(apiUrl);
var resultContent = response.Content;
var model = await resultContent.ReadAsAsync<dynamic>();
return Ok(model);
}
}

The default SSL connection idle timeout of HttpClient are 2 mins. So, after that you have to re handshake with the server. This maybe the root cause.
You could follow this article(https://www.stevejgordon.co.uk/httpclient-connection-pooling-in-dotnet-core) to extend the timeout. But the testing result from my side, I could only extend to 5 mins. After that, the connection will be closed.

Difference between HttpClient PostAsync and SendAsync

Working on a project where a WPF front end, and trying to get a handle on async calls to HttpClient and I've been going around and around trying to get PostAsync to work, but it routinely appears to deadlock, or at least the post response times out, even with large values for timeout, and a visible response in fiddler.
So, after a while I decided to try out a few other methods on HttpClient, and they worked, first try. No idea why.
I'm clean all the way to my WPF button with awaits, asyncs, and .ConfigureAwait(false) (I think):
Button:
private async void Generate_Suite_BTN_Click(object sender, RoutedEventArgs e)
{
await suiteBuilder.SendStarWs().ConfigureAwait(false);
}
XmlDoc Load:
internal async Task SendStarWs()
{
var xmlDoc = new XmlDocument();
xmlDoc.Load("C:\\Temp\\file.xml");
await StarWSClient.SendStarMessage(xmlDoc).ConfigureAwait(false);
}
SendMessage:
private static readonly HttpClient Client = new HttpClient {MaxResponseContentBufferSize = 1000000};
public static async Task<STARResult> SendMessage(vars)
{
var response = await SendRequestAsync(url, contentNew, Client).ConfigureAwait(false);
return new STARResult(response, hash);
}
This call '500s' against my endpoint immediately, which I'd expect:
var response = await SendRequestAsync(url, contentNew, Client).ConfigureAwait(false);
private static async Task<HttpResponseMessage> SendRequestAsync(string adaptiveUri, StringContent content, HttpClient httpClient)
{
HttpResponseMessage responseMessage = null;
try
{
responseMessage = await httpClient.SendAsync(new HttpRequestMessage(HttpMethod.Post, adaptiveUri)).ConfigureAwait(false);
}
catch (Exception ex)
{
if (responseMessage == null)
responseMessage = new HttpResponseMessage
{
StatusCode = HttpStatusCode.InternalServerError,
ReasonPhrase = $"SendRequestAsync failed: {ex.Message}"
};
}
return responseMessage;
}
The Post variant returns a TaskCancellationException, with timeout message regardless of timeout value:
var response = await PostRequestAsync(url, contentNew, Client).ConfigureAwait(false);
private static async Task<HttpResponseMessage> PostRequestAsync(string adaptiveUri, StringContent content, HttpClient httpClient)
{
HttpResponseMessage responseMessage = null;
try
{
responseMessage = await httpClient.PostAsync(adaptiveUri, content).ConfigureAwait(false);
}
catch (Exception ex)
{
if (responseMessage == null)
responseMessage = new HttpResponseMessage
{
StatusCode = HttpStatusCode.InternalServerError,
ReasonPhrase = $"PostRequestAsync failed: {ex.Message}"
};
}
return responseMessage;
}
My endpoint responds normally to our other software, so I'm pretty sure the endpoint is solid, I can't fathom why the post response is blocked, when the send isn't.

SendAsync can make any http verb request depending on how you set that property. PostAsync and similar are just convenience methods. Those convenience methods use SendAsync internally which is why when you derive a handler you only need to override SendAsync and not all of the send methods.
To your other question though:
When you use SendAsync you need to create the content and pass it. Your only sending an empty message. The 500 likely means that the api got null from the model binding and kicked you back. Just as #John commented.

Exception while posting to web API, HTTPClient already disposed

I'm posting a bytearray from an Android App in Xamarin.Forms to an .NET Core 2.0 WebAPI. However, I'm getting an exception saying that the NetworkStream already is disposed;
Code making the request;
public async Task PostImageAsync(ImageDTO image)
{
var content = new MultipartFormDataContent();
var byteArrayContent = new ByteArrayContent(image.Content);
content.Add(byteArrayContent, image.FileTile, image.FileName);
try
{
using (var httpClient = GetNewHttpClient())
{
SetBearerToken(httpClient);
var response = await httpClient.PostAsync($"{_apiUrl}/api/images/upload", content);
if (response.IsSuccessStatusCode)
{
}
else
{
}
}
}
catch (Exception e)
{
//Exception occurs here
var msg = e.GetBaseException().Message;
throw;
}
}
Code to get the HttpClient
private HttpClient GetNewHttpClient()
{
//HttpClientHandler is a global variable
var httpClient = new HttpClient(HttpClientHandler, false) {BaseAddress = new Uri(_apiUrl)};
return httpClient;
}
API Endpoint
[HttpPost]
public async Task<IActionResult> Upload(IFormFile file)
{
if (file == null || file.Length == 0) return BadRequest();
return Ok();
}
EDIT - SetBearerToken Method
private static void SetBearerToken(HttpClient client)
{
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", App.StoredToken);
}
The Exception:
cannot access a disposed object. Object name: 'System.Net.Sockets.NetworkStream'.
It feels like a really obvious mistake I'm making here, but I can't get my head around it. Anybody has any ideas?

Don't dispose objects inside async functions
A using statement in an async method is "odd" in that the Dispose
call may execute in a different thread to the one which acquired the
resource (depending on synchronization context etc) but it will still
happen... assuming the thing you're waiting for ever shows up or
fail, of course. (Just like you won't end up calling Dispose in
non-async code if your using statement contains a call to a method
which never returns.)
#jon-skeet https://stackoverflow.com/a/16566605/2228916

Don’t dispose of the HttpClient:
https://aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/
Also noticed that you set _apiUrl as the BaseAddress and prefix the url in the post. Pick one or the other.

A HttpRequest Redirect Exception in UWP Project

The code goes below
public static async Task<string> getForwardUrl(string url)
{
try
{
HttpClient client = new HttpClient();
HttpRequestMessage forwardRequest = new HttpRequestMessage();
forwardRequest.RequestUri = new Uri(url);
HttpResponseMessage Message = await client.SendAsync(forwardRequest);
return Message.RequestMessage.RequestUri.OriginalString;
}
catch (Exception ex)
{
WriteLine(ex.Message);
}
//...
}
When I run this in a uwp project, exception occurs. The message of this exception shows that the redirection request would change the safe connection to an unsafe one(After that , I checked the URL of the login page , It's https ,but the page after I logged in is http).
I find a similar question, he recommends using Windows.Web.Http instead of System.Net.Http but I get the same error message.
Thanks for your reply
EDIT:
The URL is: https://tinyurl.com /57muy (remove the space) or short a http url with tinyurl.com! The problem only occurs with a shortet http side!
Error: An error occurred while sending the request. Innermessage: Error message not found for this error

According to your description, I'd suppose you are developing a UWP app. And as you've mentioned, we got the exception here because the redirection request would change the safe connection to an unsafe one. To solve this problem, we can turn off auto-redirect and do the redirection by ourselves.
For example:
public static async Task<string> getForwardUrl(string url)
{
var handler = new System.Net.Http.HttpClientHandler();
handler.AllowAutoRedirect = false;
var client = new System.Net.Http.HttpClient(handler);
var response = await client.GetAsync(url);
if (response.StatusCode == System.Net.HttpStatusCode.Redirect || response.StatusCode == System.Net.HttpStatusCode.Moved)
{
return response.Headers.Location.AbsoluteUri;
}
return url;
}

HttpClient - A task was cancelled?

It works fine when have one or two tasks however throws an error "A task was cancelled" when we have more than one task listed.
List<Task> allTasks = new List<Task>();
allTasks.Add(....);
allTasks.Add(....);
Task.WaitAll(allTasks.ToArray(), configuration.CancellationToken);
private static Task<T> HttpClientSendAsync<T>(string url, object data, HttpMethod method, string contentType, CancellationToken token)
{
HttpRequestMessage httpRequestMessage = new HttpRequestMessage(method, url);
HttpClient httpClient = new HttpClient();
httpClient.Timeout = new TimeSpan(Constants.TimeOut);
if (data != null)
{
byte[] byteArray = Encoding.ASCII.GetBytes(Helper.ToJSON(data));
MemoryStream memoryStream = new MemoryStream(byteArray);
httpRequestMessage.Content = new StringContent(new StreamReader(memoryStream).ReadToEnd(), Encoding.UTF8, contentType);
}
return httpClient.SendAsync(httpRequestMessage).ContinueWith(task =>
{
var response = task.Result;
return response.Content.ReadAsStringAsync().ContinueWith(stringTask =>
{
var json = stringTask.Result;
return Helper.FromJSON<T>(json);
});
}).Unwrap();
}

There's 2 likely reasons that a TaskCanceledException would be thrown:
Something called Cancel() on the CancellationTokenSource associated with the cancellation token before the task completed.
The request timed out, i.e. didn't complete within the timespan you specified on HttpClient.Timeout.
My guess is it was a timeout. (If it was an explicit cancellation, you probably would have figured that out.) You can be more certain by inspecting the exception:
try
{
var response = task.Result;
}
catch (TaskCanceledException ex)
{
// Check ex.CancellationToken.IsCancellationRequested here.
// If false, it's pretty safe to assume it was a timeout.
}

I ran into this issue because my Main() method wasn't waiting for the task to complete before returning, so the Task<HttpResponseMessage> was being cancelled when my console program exited.
C# ≥ 7.1
You can make the main method asynchronous and await the task.
public static async Task Main(){
Task<HttpResponseMessage> myTask = sendRequest(); // however you create the Task
HttpResponseMessage response = await myTask;
// process the response
}
C# < 7.1
The solution was to call myTask.GetAwaiter().GetResult() in Main() (from this answer).

var clientHttp = new HttpClient();
clientHttp.Timeout = TimeSpan.FromMinutes(30);
The above is the best approach for waiting on a large request.
You are confused about 30 minutes; it's random time and you can give any time that you want.
In other words, request will not wait for 30 minutes if they get results before 30 minutes.
30 min means request processing time is 30 min.
When we occurred error "Task was cancelled", or large data request requirements.

Another possibility is that the result is not awaited on the client side. This can happen if any one method on the call stack does not use the await keyword to wait for the call to be completed.

Promoting #JobaDiniz's comment to an answer:
Do not do the obvious thing and dispose the HttpClient instance, even though the code "looks right":
async Task<HttpResponseMessage> Method() {
using (var client = new HttpClient())
return client.GetAsync(request);
}
Disposing the HttpClient instance can cause following HTTP requests started by other instances of HttpClient to be cancelled!
The same happens with C#'s new RIAA syntax; slightly less obvious:
async Task<HttpResponseMessage> Method() {
using var client = new HttpClient();
return client.GetAsync(request);
}
Instead, the correct approach is to cache a static instance of HttpClient for your app or library, and reuse it:
static HttpClient client = new HttpClient();
async Task<HttpResponseMessage> Method() {
return client.GetAsync(request);
}
(The Async() request methods are all thread safe.)

in my .net core 3.1 applications I am getting two problem where inner cause was timeout exception.
1, one is i am getting aggregate exception and in it's inner exception was timeout exception
2, other case was Task canceled exception
My solution is
catch (Exception ex)
{
if (ex.InnerException is TimeoutException)
{
ex = ex.InnerException;
}
else if (ex is TaskCanceledException)
{
if ((ex as TaskCanceledException).CancellationToken == null || (ex as TaskCanceledException).CancellationToken.IsCancellationRequested == false)
{
ex = new TimeoutException("Timeout occurred");
}
}
Logger.Fatal(string.Format("Exception at calling {0} :{1}", url, ex.Message), ex);
}

In my situation, the controller method was not made as async and the method called inside the controller method was async.
So I guess its important to use async/await all the way to top level to avoid issues like these.

I was using a simple call instead of async. As soon I added await and made method async it started working fine.
public async Task<T> ExecuteScalarAsync<T>(string query, object parameter = null, CommandType commandType = CommandType.Text) where T : IConvertible
{
using (IDbConnection db = new SqlConnection(_con))
{
return await db.ExecuteScalarAsync<T>(query, parameter, null, null, commandType);
}
}

Another reason can be that if you are running the service (API) and put a breakpoint in the service (and your code is stuck at some breakpoint (e.g Visual Studio solution is showing Debugging instead of Running)). and then hitting the API from the client code. So if the service code a paused on some breakpoint, you just hit F5 in VS.