I need to change a HttpClient.Timeout property after it made a request(s). When I try, I get an exception:
This instance has already started one or more requests. Properties can only be modified before sending the first request.
Is there any way to avoid this?
There isn't much you can do to change this. This is just default behavior in the HttpClient implementation.
The Timeout property must be set before the GetRequestStream or GetResponse method is called.
From HttpClient.Timeout Remark Section
In order to change the timeout, it would be best to create a new instance of an HttpClient.
client = new HttpClient();
client.Timeout = 20; //set new timeout
Internally the Timeout property is used to set up a CancellationTokenSource which will abort the async operation when that timeout is reached. Since some overloads of the HttpClient methods accept CancellationTokens, we can create helper methods to have a custom timeouts for specific operations:
public async Task<string> GetStringAsync(string requestUri, TimeSpan timeout)
{
using (var cts = new CancellationTokenSource(timeout))
{
HttpResponseMessage response = await _httpClient.GetAsync(requestUri, cts.Token)
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
Lack of support for custom request-level timeouts has always been a shortcoming of HttpClient in my mind. If you don't mind a small library dependency, Flurl.Http [disclaimer: I'm the author] supports this directly:
"http://api.com/endpoint".WithTimeout(30).GetJsonAsync<T>();
This is a true request-level setting; all calls to the same host use a shared HttpClient instance under the hood, and concurrent calls with different timeouts will not conflict. There's a configurable global default (100 seconds initially, same as HttpClient).
Related
I'm using Polly to handle some scenarios like request throttled and timeouts.
The policies were added directly in the Startup.cs which would be like this :
var retries = //applying the retries, let say I set to 25 times with 10s delay. Total 250s.
serviceCollection
.AddHttpClient<IApplicationApi, ApplicationApi>()
.AddPolicyHandler((services, request) => GetRetryPolicy<ApplicationApi>(retries, services));
The Policy:
static IAsyncPolicy<HttpResponseMessage> GetRetryPolicy<T>(List<TimeSpan> retries, IServiceProvider services)
{
return HttpPolicyExtensions
.HandleTransientHttpError()
.OrResult(msg => msg.StatusCode == System.Net.HttpStatusCode.TooManyRequests)
.WaitAndRetryAsync(retries,
onRetry: (outcome, timespan, retryAttempt, context) =>
{
//do some logging
}
}
In ApplicationApi.cs do something like this:
private readonly HttpClient _httpClient;
public ApplicationApi(HttpClient httpClient)
{
_httpClient = httpClient;
}
public void CallApi()
{
var url = "https://whateverurl.com/someapi"
using (var request = new HttpRequestMessage(HttpMethod.Get, url))
{
var response = await _httpClient.SendAsync(request);
var respMessage = await
response.Content.ReadAsStringAsync();
}
}
Now let say I don't specify the HttpClient.Timeout, which then will use default timeout : 100s.
Now I have a problem with heavy throttling. Polly will retry until the throttling resolved, or it reach the max retry.
But, the program will thrown an exception on the 10th retry since it already more than 100s elapsed on httpclient since the first request it got throttled.
Seems like the first http request that got throttled still on and not closed, or I may be wrong.
What causing this? Is it a normal behavior of Polly retries?
How can I make it close the connection on each retries so I don't have to set a very high HttpClient.Timeout value.
I also implemented the Polly timeout policy to cut request if more than some specified second then retry until it succeed. But the Polly behavior still like this. So I need to set httpclient timeout > total elapsed time on retries
**UPDATE
Code updated. So I just realized there's using statement for the request.
***UPDATE
I've created a repo that reproduce the behavior here : https://github.com/purnadika/PollyTestWebApi
The short answer is that your observed behaviour is due to fact how AddPolicyHandler and PolicyHttpMessageHandler work.
Whenever you register a new Typed HttpClient without any policy (.AddHttpClient) then you basically create a new HttpClient like this:
var handler = new HttpClientHandler();
var client = new HttpClient(handler);
Of course it is much more complicated, but from our topic perspective it works like that.
If you register a new Typed HttpClient with a policy (.AddHttpClient().AddPolicyHandler()) then you create a new HttpClient like this
var handler = new PolicyHttpMessageHandler(yourPolicy);
handler.InnerHandler = new HttpClientHandler();
var client = new HttpClient(handler);
So the outer handler will be the Polly's MessageHandler and the inner is the default ClientHandler.
Polly's MessageHandler has the following documentation comment:
/// <para>
/// Take care when using policies such as Retry or Timeout together as HttpClient provides its own timeout via
/// <see cref="HttpClient.Timeout"/>. When combining Retry and Timeout, <see cref="HttpClient.Timeout"/> will act as a
/// timeout across all tries; a Polly Timeout policy can be configured after a Retry policy in the configuration sequence,
/// to provide a timeout-per-try.
/// </para>
By using the AddPolicyHandler the HttpClient's Timeout will act as a global timeout.
The solution
There is workaround, namely avoiding the usage of AddPolicyHandler.
So, rather than decorating your Typed Client at the registration time you can decorate only the specific HttpClient method call inside your typed client.
Here is a simplified example based on your dummy project:
ConfigureServices
services.AddHttpClient<IApplicationApi, ApplicationApi>(client => client.Timeout = TimeSpan.FromSeconds(whateverLowValue));
_MainRequest
var response = await GetRetryPolicy().ExecuteAsync(async () => await _httpClient.GetAsync(url));
Here I would like to emphasize that you should prefer GetAsync over SendAsync since the HttpRequestMessage can not be reused.
So, if you would write the above code like this
using (var request = new HttpRequestMessage(HttpMethod.Get, url))
{
var response = await GetRetryPolicy().ExecuteAsync(async () => await _httpClient.SendAsync(request));
}
then you would receive the following exception:
InvalidOperationException: The request message was already sent. Cannot send the same request message multiple times.
So, with this workaround the HttpClient's Timeout will not act as a global / overarching timeout over the retry attempts.
Polly works to retry the top-level HttpClient request, so HttpClient's timeout applies to all retries. That's the whole point of using Polly, to retry requests in a way that's transparent to the top-level code.
If retrying for over a minute failed to work, the retry policy isn't good enough. Retrying over and over with a fixed delay will only result in more 429 responses, as all the requests that failed will be retried at the same time. This will result in wave after wave of identical requests hitting the server, resulting in 429s once again.
To avoid this, exponential backoff and jitter are used to introduce an increasing random delay to retries.
From the linked sample:
var delay = Backoff.DecorrelatedJitterBackoffV2(
medianFirstRetryDelay: TimeSpan.FromSeconds(1),
retryCount: 5);
var retryPolicy = Policy
.Handle<FooException>()
.WaitAndRetryAsync(delay);
The Polly page for the Jitter strategy explain how this works. The distribution graph of delays shows that even with 5 retries, the retry intervals don't clamp together.
This means there's less chance of multiple HttpClient calls retrying at the same time, resulting in renewed throttling
We use System.Net.Http.HttpClientfor calls between microservices inside k8s.
OS Linux(inside docker)
dotnet TargetFramework: netcoreapp2.1
server: Kestrel
protocol: http
Few days ago we noticed very strange behaviour of http calls:
some calls between microservice(near 2-3%) failed with error
System.OperationCanceledException: The operation was canceled.
at System.Net.Http.HttpClient.HandleFinishSendAsyncError(Exception e, CancellationTokenSource cts)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at OurCode.HttpServiceClient.GetStringResponseAsync(String methodUri, HttpMethod httpMethod)
...another our code...
after our timeout for http calls(it is 3 sec). But there was no logs about call inside callee service.
We enabled packetbeat for tracing http requests and also noticed, that no any http requests from caller service to callee service was executed.
CPU, memory and network for this services was OK all the time.
Simplified version of our code for http calls looks like:
public async Task<string> GetStringResponseAsync(String methodUri, HttpMethod httpMethod)
{
int timeoutInMilliseconds = 3000;
var tokenSource = new CancellationTokenSource();
var rm = new HttpRequestMessage(httpMethod, methodUri);
Task<HttpResponseMessage> httpTask = HttpClient.SendAsync(rm, tokenSource.Token);
tokenSource.CancelAfter(timeoutInMilliseconds);
HttpResponseMessage response = await httpTask;
await EnsureSuccessStatusCode(response);
return await response.Content.ReadAsStringAsync();
}
Any ideas about what problem can cause this strange behaviour without http request through network, and what can I do for further investigation?
It just meant that the web service did not respond.
HttpClient throws a TaskCanceledException, (which inherits from OperationCanceledException) when the timeout elapses. It is not intuitive and doesn't make any sense to me (and others), but that's what it does unfortunately.
There is some discussion about it here (a few people mentioned some workarounds to distinguish timeout from a true cancel, if you care).
I don't know if this will help anyone in the future but I had the same issues. Turns out the server I was hosting on was not automatically using the proxy address by default. I ended up having to add the following to my code:
using (var handler = new WinHttpHandler()){
handler.WindowsProxyUsePolicy = WindowsProxyUsePolicy.UseCustomProxy;
handler.Proxy = new WebProxy(server, port);
//the rest of your code.....
}
This manually added the proxy server information and my console app began working as expected.
I'm trying to set a default timeout for my HttpClient calls to 5 seconds.
I've done this via CancellationTokenSource.
Here's the pertinent bit of code:
var cancellationToken = new CancellationTokenSource();
cancellationToken.CancelAfter(TimeSpan.FromSeconds(5));
var result = _httpClient.SendAsync(request, cancellationToken.Token);
Works as i expected in terms of the calling code getting a "Task was cancelled" error (i tested in a .NET 4.7 console app), but i noticed in Fiddler the request was still running for 1 minute, until it finally gave up:
Can someone explain this behaviour?
I would expect the underlying request to also get cancelled when the cancellation is triggered.
_httpClient is instantiated like: new HttpClient { BaseAddress = baseAddress }
I know there's the the Timeout setting, but not sure if I should be using that or cancellation tokens? My guess is Timeout is for the non-async/await cases?
As Damien said in the comments, HttpClient re-uses connections as much as possible, hence the reason why the connection is not closed on cancel.
When canceling a request like that, the HttpClient will just stop sending/receiving data to/from the other end. It will not send anything to inform the other end that it was cancelled. So the timeout you see of 1 minute depends on the behavior of the other end of your connection.
Also, if you want to cancel each request after 5 seconds, you can as well set the Timeout property of _httpClient to TimeSpan.FromSeconds(5). The behavior will be exactly the same (a TaskCanceledException will be thrown if the other end doesn't respond within 5 seconds).
If anyone is interested, you can try the following approach to applying your own timeout per HttpClient request. It seems to work for me, restricting the SendAsync() to 2 seconds and returning immediately when the timeout occurs:
private async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, TimeSpan? timeout = null)
{
if (timeout is null)
{
return await _httpClient.SendAsync(request);
}
else
{
using (var cts = new CancellationTokenSource(timeout.Value))
{
var sendTask = _httpClient.SendAsync(request);
while (!sendTask.IsCompleted)
{
cts.Token.ThrowIfCancellationRequested();
await Task.Delay(10).ConfigureAwait(false);
}
return await sendTask.ConfigureAwait(false);
}
}
}
I am trying to set a timeout for a special request which will take a long time to process. Because of this, I am trying to set the timeout, like this:
client.RequestFilter = r => {
r.Timeout = 1000000;
r.ReadWriteTimeout = 1000000;
}
However, these settings seem to have no effect; the request still times out in about 30 seconds. Is there some hack I can use to set the timeout properly ?
ETA: The response I'm receiving is a stream; I do it like this:
var stream = client.Send<Stream>(requestDto);
Is there a better way ?
ServiceStack's Service Clients is just a wrapper around HttpWebRequest so your code ends up setting the HttpWebRequest Timeout and ReadWriteTimeout properties directly.
The Request Filter gives you direct access to the HttpWebRequest instance used and setting the Timeout properties should work as expected. Other than that the only class that can modify behavior of .NET's HttpWebRequest is System.Net.ServicePointManager which lets you configure some properties like DefaultConnectionLimit and DnsRefreshTimeout, etc. But there's no additional Request Timeout properties.
The alternative solution you can try is to use ServiceStack's JsonHttpClient which as it's built on Microsoft's newer HttpClient library, you may have better luck with it. Although it's recommended to use the Async API's since the Sync API's are just blocking on the HttpClient's underlying Async API's.
For the API call itself, you should access the stream in a using block, e.g:
using (var stream = client.Send<Stream>(requestDto))
{
}
When making a REST call, the usual approach with C# is:
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://localhost:9000/");
client.SendAsync(....);
}
But what if I want every single outbound request to have a custom header, without manual coding? With a WCF proxy I can achieve that result by adding a global endpoint behavior (e.g. IMessageInspector) which will hook every outbound call.
With HttpClient, I don't think such a hook point exists. Perhaps the best I can do is to create an extension method for HttpClient which automatically adds the header. Unfortunately, that means that every developer must still voluntarily comply and remember to manually invoke the extension method.
Any solution I'm overlooking?
p.s. I understand that REST is supposed to be light-weight, and so much of the abstraction of WCF is jettisoned. Still, no harm in asking...
You could use a custom HttpMessageHandler:
public class YourServiceMessageHandler : HttpClientHandler
{
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request, CancellationToken cancellationToken)
{
// Add header to request here
return base.SendAsync(request, cancellationToken);
}
}
And pass it to the HttpClient constructor:
HttpClient client = new HttpClient(new YourServiceMessageHandler());
No hooks like that, but if you are writing a HTTP API, you can just hide the entire uploading process in your class so the users of your API only call something like SendData(dataPackage);, and instead of using HttpRequest in your extension as a parent, you just create an instance of it so others don't see what you are doing.