It seems that GetResponseAsync does not accept cancellationToken in Async/Await. So the question is how can I cancel the below procedure, provided I need to collect Cookies from response:
using (HttpWebResponse response = (HttpWebResponse) await request.GetResponseAsync())
{
cookies.Add(response.Cookies);
}
An alternative code to achieve the above is also welcome.
Something like this should work (untested):
public static class Extensions
{
public static async Task<HttpWebResponse> GetResponseAsync(this HttpWebRequest request, CancellationToken ct)
{
using (ct.Register(() => request.Abort(), useSynchronizationContext: false))
{
var response = await request.GetResponseAsync();
ct.ThrowIfCancellationRequested();
return (HttpWebResponse)response;
}
}
}
In theory, if cancellation is requested on ct and request.Abort is invoked, await request.GetResponseAsync() should throw a WebException. IMO though, it's always a good idea to check for cancellation explicitly when consuming the result, to mitigate race conditions, so I call ct.ThrowIfCancellationRequested().
Also, I assume that request.Abort is thread-safe (can be called from any thread), so I use useSynchronizationContext: false (I haven't verified that).
[UPDATED] to address the OP's comment on how to differentiate between WebException caused by cancellation and any other error. This is how it can be done, so TaskCanceledException (derived from OperationCanceledException) will be correctly thrown upon cancellation:
public static class Extensions
{
public static async Task<HttpWebResponse> GetResponseAsync(this HttpWebRequest request, CancellationToken ct)
{
using (ct.Register(() => request.Abort(), useSynchronizationContext: false))
{
try
{
var response = await request.GetResponseAsync();
return (HttpWebResponse)response;
}
catch (WebException ex)
{
// WebException is thrown when request.Abort() is called,
// but there may be many other reasons,
// propagate the WebException to the caller correctly
if (ct.IsCancellationRequested)
{
// the WebException will be available as Exception.InnerException
throw new OperationCanceledException(ex.Message, ex, ct);
}
// cancellation hasn't been requested, rethrow the original WebException
throw;
}
}
}
}
public static async Task<T> WithCancellation<T>(this Task<T> task, CancellationToken cancellationToken, Action action, bool useSynchronizationContext = true)
{
using (cancellationToken.Register(action, useSynchronizationContext))
{
try
{
return await task;
}
catch (Exception ex)
{
if (cancellationToken.IsCancellationRequested)
{
// the Exception will be available as Exception.InnerException
throw new OperationCanceledException(ex.Message, ex, cancellationToken);
}
// cancellation hasn't been requested, rethrow the original Exception
throw;
}
}
}
Now you can use your cancellation token on any cancelable async method. For example WebRequest.GetResponseAsync:
var request = (HttpWebRequest)WebRequest.Create(url);
using (var response = await request.GetResponseAsync())
{
. . .
}
will become:
var request = (HttpWebRequest)WebRequest.Create(url);
using (WebResponse response = await request.GetResponseAsync().WithCancellation(CancellationToken.None, request.Abort, true))
{
. . .
}
See example http://pastebin.com/KauKE0rW
Related
Let's say I have a simple method:
private void MyMethod()
{
try {
myService.Do();
} catch (MyException ex) {}
}
I have a service that uses HttpClient, and I do:
public void Do()
{
var response = client.GetAsync(url).Result; //Alas it's not async till now
}
And now I'm implementing a DelegatingHandler and I override SendAsync:
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
return await base.SendAsync(request, cancellationToken)
.ContinueWith<HttpResponseMessage>(task =>
{
HttpResponseMessage response = task.Result;
if (response.StatusCode == HttpStatusCode.NotFound)
{
throw new MyException();
}
return response;
}).ConfigureAwait(false);
}
Everything works as expected until a MyException is thrown here. The exception bubbles up to the caller, however, the exception is an AggregrateException.
Is there a way to throw the actual exception itself so that no major application-wide code-change is required?
When exception is thrown in task then using Result property you will get AggregateException where the real exception is in InnerException property of this AggregateException object (your exception is wrapped by AggregateException).
To get the true exception (unwrapped exception) you can use GetAwaiter().GetResult():
var result = taskThatThrowsException.GetAwaiter().GetResult();
You can also use Result property but then you should specify some condition for exception handling;
try
{
var result = taskThatThrowsException.Result;
}
catch (AggregateException ex) when (ex.InnerException is MyException myException)
{
// use myException - it is your true unwrapped exception
}
But you should NOT block asynchronous code - you should asynchronously wait - use await and you will get your true unwrapped exception also:
var result = await taskThatThrowsException;
AggregrateException is a consolidation of many exceptions, as an async method can throw many exceptions uppon its execution .net uses this class to return all of them.
To access the inner exceptions it has a property called InnerExceptions, it's a readonly collection that contains all the thrown ones.
I am building a function that given an HttpContent Object, will issues request and retry on failure. However I get exceptions saying that HttpContent Object is disposed after issuing the request. Is there anyway to copy or duplicate the HttpContent Object so that I can issue multiple requests.
public HttpResponseMessage ExecuteWithRetry(string url, HttpContent content)
{
HttpResponseMessage result = null;
bool success = false;
do
{
using (var client = new HttpClient())
{
result = client.PostAsync(url, content).Result;
success = result.IsSuccessStatusCode;
}
}
while (!success);
return result;
}
// Works with no exception if first request is successful
ExecuteWithRetry("http://www.requestb.in/xfxcva" /*valid url*/, new StringContent("Hello World"));
// Throws if request has to be retried ...
ExecuteWithRetry("http://www.requestb.in/badurl" /*invalid url*/, new StringContent("Hello World"));
(Obviously I don't try indefinitely but the code above is essentially what i want).
It yields this exception
System.AggregateException: One or more errors occurred. ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Http.StringContent'.
at System.Net.Http.HttpContent.CheckDisposed()
at System.Net.Http.HttpContent.CopyToAsync(Stream stream, TransportContext context)
at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at System.Threading.Tasks.Task`1.get_Result()
at Submission#8.ExecuteWithRetry(String url, HttpContent content)
Is there anyway to duplicate an HttpContent Object or reuse it?
Instead of implementing retry functionality that wraps the HttpClient, consider constructing the HttpClient with a HttpMessageHandler that performs the retry logic internally. For example:
public class RetryHandler : DelegatingHandler
{
// Strongly consider limiting the number of retries - "retry forever" is
// probably not the most user friendly way you could respond to "the
// network cable got pulled out."
private const int MaxRetries = 3;
public RetryHandler(HttpMessageHandler innerHandler)
: base(innerHandler)
{ }
protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
HttpResponseMessage response = null;
for (int i = 0; i < MaxRetries; i++)
{
response = await base.SendAsync(request, cancellationToken);
if (response.IsSuccessStatusCode) {
return response;
}
}
return response;
}
}
public class BusinessLogic
{
public void FetchSomeThingsSynchronously()
{
// ...
// Consider abstracting this construction to a factory or IoC container
using (var client = new HttpClient(new RetryHandler(new HttpClientHandler())))
{
myResult = client.PostAsync(yourUri, yourHttpContent).Result;
}
// ...
}
}
ASP.NET Core 2.1 Answer
ASP.NET Core 2.1 added support for Polly directly. Here UnreliableEndpointCallerService is a class which accepts a HttpClient in its constructor. Failed requests will retry with an exponential back-off so that the next retry takes place in an exponentially longer time after the previous one:
services
.AddHttpClient<UnreliableEndpointCallerService>()
.AddTransientHttpErrorPolicy(
x => x.WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(3, retryAttempt)));
Also, consider reading my blog post "Optimally Configuring HttpClientFactory".
Other Platforms Answer
This implementation uses Polly to retry with an exponential back-off so that the next retry takes place in an exponentially longer time after the previous one. It also retries if a HttpRequestException or TaskCanceledException is thrown due to a timeout. Polly is much easier to use than Topaz.
public class HttpRetryMessageHandler : DelegatingHandler
{
public HttpRetryMessageHandler(HttpClientHandler handler) : base(handler) {}
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken) =>
Policy
.Handle<HttpRequestException>()
.Or<TaskCanceledException>()
.OrResult<HttpResponseMessage>(x => !x.IsSuccessStatusCode)
.WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(3, retryAttempt)))
.ExecuteAsync(() => base.SendAsync(request, cancellationToken));
}
using (var client = new HttpClient(new HttpRetryMessageHandler(new HttpClientHandler())))
{
var result = await client.GetAsync("http://example.com");
}
The current answers won't work as expected in all cases, specifically in the very common case of request timeout (see my comments there).
In addition, they implement a very naive retry strategy - many times you'd want something a bit more sophosticated, such as exponential backoff (which is the default in the Azure Storage Client API).
I stumbled upon TOPAZ while reading a related blog post (also offering the misguided internal retry approach). Here's what I came up with:
// sample usage: var response = await RequestAsync(() => httpClient.GetAsync(url));
Task<HttpResponseMessage> RequestAsync(Func<Task<HttpResponseMessage>> requester)
{
var retryPolicy = new RetryPolicy(transientErrorDetectionStrategy, retryStrategy);
//you can subscribe to the RetryPolicy.Retrying event here to be notified
//of retry attempts (e.g. for logging purposes)
return retryPolicy.ExecuteAsync(async () =>
{
HttpResponseMessage response;
try
{
response = await requester().ConfigureAwait(false);
}
catch (TaskCanceledException e) //HttpClient throws this on timeout
{
//we need to convert it to a different exception
//otherwise ExecuteAsync will think we requested cancellation
throw new HttpRequestException("Request timed out", e);
}
//assuming you treat an unsuccessful status code as an error
//otherwise just return the respone here
return response.EnsureSuccessStatusCode();
});
}
Note the requester delegate parameter. It should not be an HttpRequestMessage since you can't send the same request multiple times. As for the strategies, that depends on your use case. For example, a transient error detection strategy could be as simple as:
private sealed class TransientErrorCatchAllStrategy : ITransientErrorDetectionStrategy
{
public bool IsTransient(Exception ex)
{
return true;
}
}
As for the retry strategy, TOPAZ offers three options:
FixedInterval
Incremental
ExponentialBackoff
For example, here's the TOPAZ equivalent of what the Azure Client Storage Library uses for default:
int retries = 3;
var minBackoff = TimeSpan.FromSeconds(3.0);
var maxBackoff = TimeSpan.FromSeconds(120.0);
var deltaBackoff= TimeSpan.FromSeconds(4.0);
var strategy = new ExponentialBackoff(retries, minBackoff, maxBackoff, deltaBackoff);
For more information see http://msdn.microsoft.com/en-us/library/hh680901(v=pandp.50).aspx
EDIT Note that if your request contains an HttpContent object, you'll have to regenerate it every time as that will be disposed by HttpClient as well (thanks for catching that Alexandre Pepin). For example () => httpClient.PostAsync(url, new StringContent("foo"))).
Duplicating the StringContent isn't probably the best idea. But simple modification could fix the problem. Just modify the function and create the StringContent object inside of the loop, something like:
public HttpResponseMessage ExecuteWithRetry(string url, string contentString)
{
HttpResponseMessage result = null;
bool success = false;
using (var client = new HttpClient())
{
do
{
result = client.PostAsync(url, new StringContent(contentString)).Result;
success = result.IsSuccessStatusCode;
}
while (!success);
}
return result;
}
and then call it
ExecuteWithRetry("http://www.requestb.in/xfxcva" /*valid url*/, "Hello World");
This is what I achieved using polly.
nuget
https://www.nuget.org/packages/Microsoft.Extensions.Http.Polly
https://www.nuget.org/packages/Polly
using Polly;
using Polly.Extensions.Http;
//// inside configure service
services.AddHttpClient("RetryHttpClient", c =>
{
c.BaseAddress = new Uri($"{configuration["ExternalApis:MyApi"]}/");
c.DefaultRequestHeaders.Add("Accept", "application/json");
c.Timeout = TimeSpan.FromMinutes(5);
c.DefaultRequestHeaders.ConnectionClose = true;
}).AddPolicyHandler(GetRetryPolicy());
//// add this method to give retry policy
private static IAsyncPolicy<HttpResponseMessage> GetRetryPolicy()
{
return HttpPolicyExtensions
//// 408,5xx
.HandleTransientHttpError()
//// 404
.OrResult(msg => msg.StatusCode == HttpStatusCode.NotFound)
//// 401
.OrResult(msg => msg.StatusCode == HttpStatusCode.Unauthorized)
//// Retry 3 times, with wait 1,2 and 4 seconds.
.WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)));
}
This builds off the accepted answer but adds the ability to pass in the amount of retries, plus the ability to add non-blocking delays / wait time to each request. It also uses a try catch to ensure the retry continues to happen after an exception has occurred. And last, I added code to break out of the loop in the case of BadRequests, you don't want to resend the same bad request multiple times.
public class HttpRetryHandler : DelegatingHandler
{
private int MaxRetries;
private int WaitTime;
public HttpRetryHandler(HttpMessageHandler innerHandler, int maxRetries = 3, int waitSeconds = 0)
: base(innerHandler)
{
MaxRetries = maxRetries;
WaitTime = waitSeconds * 1000;
}
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
HttpResponseMessage response = null;
for (int i = 0; i < MaxRetries; i++)
{
try
{
response = await base.SendAsync(request, cancellationToken);
if (response.IsSuccessStatusCode)
{
return response;
}
else if(response.StatusCode == HttpStatusCode.BadRequest)
{
// Don't reattempt a bad request
break;
}
}
catch
{
// Ignore Error As We Will Attempt Again
}
finally
{
response.Dispose();
}
if(WaitTime > 0)
{
await Task.Delay(WaitTime);
}
}
return response;
}
}
With RestEase And Task, on retry with httpClient reused in many call (singleton) it frezze and throw TaskCanceledException.
To fix this whe need to Dispose() the failed response before retry
public class RetryHandler : DelegatingHandler
{
// Strongly consider limiting the number of retries - "retry forever" is
// probably not the most user friendly way you could respond to "the
// network cable got pulled out."
private const int MaxRetries = 3;
public RetryHandler(HttpMessageHandler innerHandler)
: base(innerHandler)
{ }
protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
HttpResponseMessage response = null;
for (int i = 0; i < MaxRetries; i++)
{
response = await base.SendAsync(request, cancellationToken).ConfigureAwait(false);
if (response.IsSuccessStatusCode) {
return response;
}
response.Dispose();
}
return response;
}
}
I tried it and worked while using unit and integration tests. However, it stuck when I actually called from REST URL. I found this interesting post which explains why it gets stuck at this line.
response = await base.SendAsync(request, cancellationToken);
The fix to this is that you have .ConfigureAwait(false) added at the end.
response = await base.SendAsync(request, token).ConfigureAwait(false);
I also added create linked token part there like this.
var linkedToken = cancellationToken.CreateLinkedSource();
linkedToken.CancelAfter(new TimeSpan(0, 0, 5, 0));
var token = linkedToken.Token;
HttpResponseMessage response = null;
for (int i = 0; i < MaxRetries; i++)
{
response = await base.SendAsync(request, token).ConfigureAwait(false);
if (response.IsSuccessStatusCode)
{
return response;
}
}
return response;
You also refer to Building a Transient Retry Handler for the .NET HttpClient.
Visit refer to KARTHIKEYAN VIJAYAKUMAR post.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Data.SqlClient;
using System.Net.Http;
using System.Threading;
using System.Diagnostics;
using System.Net;
using Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling;
namespace HttpClientRetyDemo
{
class Program
{
static void Main(string[] args)
{
var url = "http://RestfulUrl";
var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, url);
var handler = new RetryDelegatingHandler
{
UseDefaultCredentials = true,
PreAuthenticate = true,
Proxy = null
};
HttpClient client = new HttpClient(handler);
var result = client.SendAsync(httpRequestMessage).Result.Content
.ReadAsStringAsync().Result;
Console.WriteLine(result.ToString());
Console.ReadKey();
}
}
/// <summary>
/// Retry Policy = Error Detection Strategy + Retry Strategy
/// </summary>
public static class CustomRetryPolicy
{
public static RetryPolicy MakeHttpRetryPolicy()
{
// The transient fault application block provides three retry policies
// that you can use. These are:
return new RetryPolicy(strategy, exponentialBackoff);
}
}
/// <summary>
/// This class is responsible for deciding whether the response was an intermittent
/// transient error or not.
/// </summary>
public class HttpTransientErrorDetectionStrategy : ITransientErrorDetectionStrategy
{
public bool IsTransient(Exception ex)
{
if (ex != null)
{
HttpRequestExceptionWithStatus httpException;
if ((httpException = ex as HttpRequestExceptionWithStatus) != null)
{
if (httpException.StatusCode == HttpStatusCode.ServiceUnavailable)
{
return true;
}
else if (httpException.StatusCode == HttpStatusCode.MethodNotAllowed)
{
return true;
}
return false;
}
}
return false;
}
}
/// <summary>
/// The retry handler logic is implementing within a Delegating Handler. This has a
/// number of advantages.
/// An instance of the HttpClient can be initialized with a delegating handler making
/// it super easy to add into the request pipeline.
/// It also allows you to apply your own custom logic before the HttpClient sends the
/// request, and after it receives the response.
/// Therefore it provides a perfect mechanism to wrap requests made by the HttpClient
/// with our own custom retry logic.
/// </summary>
class RetryDelegatingHandler : HttpClientHandler
{
public RetryPolicy retryPolicy { get; set; }
public RetryDelegatingHandler()
: base()
{
retryPolicy = CustomRetryPolicy.MakeHttpRetryPolicy();
}
protected async override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
HttpResponseMessage responseMessage = null;
var currentRetryCount = 0;
//On Retry => increments the retry count
retryPolicy.Retrying += (sender, args) =>
{
currentRetryCount = args.CurrentRetryCount;
};
try
{
await retryPolicy.ExecuteAsync(async () =>
{
responseMessage = await base.SendAsync(request, cancellationToken)
.ConfigureAwait(false);
if ((int)responseMessage.StatusCode > 500)
{
// When it fails after the retries, it would throw the exception
throw new HttpRequestExceptionWithStatus(
string.Format("Response status code {0} indicates server error",
(int)responseMessage.StatusCode))
{
StatusCode = responseMessage.StatusCode,
CurrentRetryCount = currentRetryCount
};
}// returns the response to the main method(from the anonymous method)
return responseMessage;
}, cancellationToken).ConfigureAwait(false);
return responseMessage;// returns from the main method => SendAsync
}
catch (HttpRequestExceptionWithStatus exception)
{
if (exception.CurrentRetryCount >= 3)
{
//write to log
}
if (responseMessage != null)
{
return responseMessage;
}
throw;
}
catch (Exception)
{
if (responseMessage != null)
{
return responseMessage;
}
throw;
}
}
}
/// <summary>
/// Custom HttpRequestException to allow include additional properties on my exception,
/// which can be used to help determine whether the exception is a transient
/// error or not.
/// </summary>
public class HttpRequestExceptionWithStatus : HttpRequestException
{
public HttpStatusCode StatusCode { get; set; }
public int CurrentRetryCount { get; set; }
public HttpRequestExceptionWithStatus()
: base() { }
public HttpRequestExceptionWithStatus(string message)
: base(message) { }
public HttpRequestExceptionWithStatus(string message, Exception inner)
: base(message, inner) { }
}
}
i have almost the same issue.
HttpWebRequest queueing library, which guarantees request delivery
I just updated (see EDIT3) my approach to avoid crashes, but i still need general mechanism to guarantee message delivery (or re-delivery in case message was not delivered).
I have the same problem and solved . it's about the "StringContent" / "HttpContent"
Please check Amogh Natu's Blog which help me to solve this problem
The problem with this code is, when the first call to PostAsync is
made and it fails, the httpContent object is disposed. This is as
designed in the HttpClient class. Refer the comment in this method.
Although this seems odd, they intent to do this so that the user
doesn’t have to do this explicitly and also to avoid the same request
being Posted more than once.
So what happens is, when the first call fails, httpContent is
disposed, then as we have retry mechanism, it tries to make the post
call again, now with a disposed object and hence this time, the call
fails with an ObjectDisposedException.
An easy way to resolve this is to NOT use a variable to store
httpContent and instead, create http content directly while making the
call. Something like this.
http://amoghnatu.net/2017/01/12/cannot-access-a-disposed-object-system-net-http-stringcontent-while-having-retry-logic/
Adding an answer that uses both Polly + Retry policy + per-retry Timeout policy, as top answer does not address that:
Policy
.Handle<HttpRequestException>()
.Or<TaskCanceledException>()
.Or<TimeoutRejectedException>()
.OrResult<HttpResponseMessage>(x => !x.IsSuccessStatusCode)
.WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(3, retryAttempt)))
.WrapAsync(
Policy.TimeoutAsync(TimeSpan.FromSeconds(1), delegate (Context ctx, TimeSpan timeSpan, Task task)
{
// Do some on-timeout action
return Task.CompletedTask;
})
)
.ExecuteAsync(() =>
{
return httpclient.PostAsync(url, httpRequest);
});
//Could retry say 5 times
HttpResponseMessage response;
int numberOfRetry = 0;
using (var httpClient = new HttpClient())
{
do
{
response = await httpClient.PostAsync(uri, content);
numberOfRetry++;
} while (response.IsSuccessStatusCode == false | numberOfRetry < 5);
}
return response;
.........
My problem is the following:
On the UI thread I have a button event, from where I call a service method:
private async void RefreshObjectsButton_Click(object sender, EventArgs e)
{
var objectService = new ObjectService();
var objects = await objectService.GetObjects(UserInfo.Token);
}
The service method is:
public class ObjectService : ServiceClientBase, IObjectService
{
public async Task<ObservableCollection<ObjectViewModel>> GetObjects(string token)
{
var response = await GetAsync<ObservableCollection<HookViewModel>>("uri_address", token);
return response;
}
}
And the GetAsync method which is implemented in ServiceClientBase:
public async Task<T> GetAsync<T>(string uri, string token)
{
using (var client = CreateClient())
{
try
{
HttpResponseMessage response = new HttpResponseMessage();
response = await client.GetAsync(uri);
T retVal = default(T);
if (response.IsSuccessStatusCode)
{
retVal = await response.Content.ReadAsAsync<T>();
}
return retVal;
}
catch (Exception ex)
{
//TO DO log exception
return default(T);
}
}
}
When execution reaches GetAsync<T>(), the request is sent, I get a result and retVal contains a list of values. However after GetAsync<T>() execution ends, GetObjects() will not continue its execution anymore. I believe it's a deadlock as explained here. However using the advices seen in the previous link didn't resolve my issue. It's clearly I'm missing something here.
Can someone explain why is this deadlock happening, and maybe provide some further advice in resolving this issue?
I have written a method which get ETag from eq. XML file on server. Am I correct wrote abort task if timeout (GetResponseAsync() doesn't have CancellationToken) and I have no other idea how to do Exception.
Here's code:
public static async Task<string> GetETagAsync(Uri feedLink)
{
const int millisecondsTimeout = 2500;
WebRequest webRequest = WebRequest.Create(feedLink);
webRequest.Method = "HEAD";
try
{
Task<WebResponse> webResponse = webRequest.GetResponseAsync();
if (await Task.WhenAny(webResponse, Task.Delay(millisecondsTimeout)) == webResponse)
{
using (var result = webResponse.Result)
{
return result.Headers["ETag"];
}
}
else
{
webRequest.Abort();
return null;
}
}
catch (Exception)
{
return null;
}
}
Edit
I have made some changes. Rewrite exceptions and use class from this topic: GetResponseAsync does not accept cancellationToken
Code:
public static async Task<string> GetETagAsync(Uri feedLink)
{
const int millisecondsTimeout = 2500;
var cancellationTokenSource = new CancellationTokenSource();
WebRequest webRequest = WebRequest.Create(feedLink);
webRequest.Method = "HEAD";
try
{
Task<WebResponse> webResponse = WebRequestExtensions.GetResponseAsync(webRequest, cancellationTokenSource.Token);
if (await Task.WhenAny(webResponse, Task.Delay(millisecondsTimeout)) == webResponse)
{
using (var result = webResponse.Result)
{
return result.Headers["ETag"];
}
}
else
{
cancellationTokenSource.Cancel();
return null;
}
}
catch (AggregateException ex)
{
if (ex.InnerException is WebException)
return null;
throw;
}
}
public static class WebRequestExtensions
{
public static async Task<WebResponse> GetResponseAsync(this WebRequest request, CancellationToken cancellationToken)
{
using (cancellationToken.Register(() => request.Abort(), useSynchronizationContext: false))
{
try
{
var response = await request.GetResponseAsync();
cancellationToken.ThrowIfCancellationRequested();
return (WebResponse)response;
}
catch (WebException ex)
{
if (ex.Status == WebExceptionStatus.RequestCanceled)
{
cancellationToken.ThrowIfCancellationRequested();
}
if (cancellationToken.IsCancellationRequested)
{
throw new TaskCanceledException(ex.Message, ex);
}
throw;
}
}
}
}
Is it correct now?
You can use the WebRequestExtensions class I wrote here to resolve the problem:
https://github.com/openstacknetsdk/openstack.net/blob/master/src/corelib/Core/WebRequestExtensions.cs
This class fully supports both a CancellationToken and the WebRequest.Timeout property.
Since the method doesn't natively support any means of stopping the task, it can't be stopped. There is nothing that you can do that will prevent it from continuing to do whatever it plans to do. What you can do is ensure that this async method's task doesn't wait for it to finish. While you can just return a value, as you're doing, this seems to represent an exceptional case; you should probably be throwing an exception to indicate that the operation timed out.
I am building a function that given an HttpContent Object, will issues request and retry on failure. However I get exceptions saying that HttpContent Object is disposed after issuing the request. Is there anyway to copy or duplicate the HttpContent Object so that I can issue multiple requests.
public HttpResponseMessage ExecuteWithRetry(string url, HttpContent content)
{
HttpResponseMessage result = null;
bool success = false;
do
{
using (var client = new HttpClient())
{
result = client.PostAsync(url, content).Result;
success = result.IsSuccessStatusCode;
}
}
while (!success);
return result;
}
// Works with no exception if first request is successful
ExecuteWithRetry("http://www.requestb.in/xfxcva" /*valid url*/, new StringContent("Hello World"));
// Throws if request has to be retried ...
ExecuteWithRetry("http://www.requestb.in/badurl" /*invalid url*/, new StringContent("Hello World"));
(Obviously I don't try indefinitely but the code above is essentially what i want).
It yields this exception
System.AggregateException: One or more errors occurred. ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Http.StringContent'.
at System.Net.Http.HttpContent.CheckDisposed()
at System.Net.Http.HttpContent.CopyToAsync(Stream stream, TransportContext context)
at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at System.Threading.Tasks.Task`1.get_Result()
at Submission#8.ExecuteWithRetry(String url, HttpContent content)
Is there anyway to duplicate an HttpContent Object or reuse it?
Instead of implementing retry functionality that wraps the HttpClient, consider constructing the HttpClient with a HttpMessageHandler that performs the retry logic internally. For example:
public class RetryHandler : DelegatingHandler
{
// Strongly consider limiting the number of retries - "retry forever" is
// probably not the most user friendly way you could respond to "the
// network cable got pulled out."
private const int MaxRetries = 3;
public RetryHandler(HttpMessageHandler innerHandler)
: base(innerHandler)
{ }
protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
HttpResponseMessage response = null;
for (int i = 0; i < MaxRetries; i++)
{
response = await base.SendAsync(request, cancellationToken);
if (response.IsSuccessStatusCode) {
return response;
}
}
return response;
}
}
public class BusinessLogic
{
public void FetchSomeThingsSynchronously()
{
// ...
// Consider abstracting this construction to a factory or IoC container
using (var client = new HttpClient(new RetryHandler(new HttpClientHandler())))
{
myResult = client.PostAsync(yourUri, yourHttpContent).Result;
}
// ...
}
}
ASP.NET Core 2.1 Answer
ASP.NET Core 2.1 added support for Polly directly. Here UnreliableEndpointCallerService is a class which accepts a HttpClient in its constructor. Failed requests will retry with an exponential back-off so that the next retry takes place in an exponentially longer time after the previous one:
services
.AddHttpClient<UnreliableEndpointCallerService>()
.AddTransientHttpErrorPolicy(
x => x.WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(3, retryAttempt)));
Also, consider reading my blog post "Optimally Configuring HttpClientFactory".
Other Platforms Answer
This implementation uses Polly to retry with an exponential back-off so that the next retry takes place in an exponentially longer time after the previous one. It also retries if a HttpRequestException or TaskCanceledException is thrown due to a timeout. Polly is much easier to use than Topaz.
public class HttpRetryMessageHandler : DelegatingHandler
{
public HttpRetryMessageHandler(HttpClientHandler handler) : base(handler) {}
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken) =>
Policy
.Handle<HttpRequestException>()
.Or<TaskCanceledException>()
.OrResult<HttpResponseMessage>(x => !x.IsSuccessStatusCode)
.WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(3, retryAttempt)))
.ExecuteAsync(() => base.SendAsync(request, cancellationToken));
}
using (var client = new HttpClient(new HttpRetryMessageHandler(new HttpClientHandler())))
{
var result = await client.GetAsync("http://example.com");
}
The current answers won't work as expected in all cases, specifically in the very common case of request timeout (see my comments there).
In addition, they implement a very naive retry strategy - many times you'd want something a bit more sophosticated, such as exponential backoff (which is the default in the Azure Storage Client API).
I stumbled upon TOPAZ while reading a related blog post (also offering the misguided internal retry approach). Here's what I came up with:
// sample usage: var response = await RequestAsync(() => httpClient.GetAsync(url));
Task<HttpResponseMessage> RequestAsync(Func<Task<HttpResponseMessage>> requester)
{
var retryPolicy = new RetryPolicy(transientErrorDetectionStrategy, retryStrategy);
//you can subscribe to the RetryPolicy.Retrying event here to be notified
//of retry attempts (e.g. for logging purposes)
return retryPolicy.ExecuteAsync(async () =>
{
HttpResponseMessage response;
try
{
response = await requester().ConfigureAwait(false);
}
catch (TaskCanceledException e) //HttpClient throws this on timeout
{
//we need to convert it to a different exception
//otherwise ExecuteAsync will think we requested cancellation
throw new HttpRequestException("Request timed out", e);
}
//assuming you treat an unsuccessful status code as an error
//otherwise just return the respone here
return response.EnsureSuccessStatusCode();
});
}
Note the requester delegate parameter. It should not be an HttpRequestMessage since you can't send the same request multiple times. As for the strategies, that depends on your use case. For example, a transient error detection strategy could be as simple as:
private sealed class TransientErrorCatchAllStrategy : ITransientErrorDetectionStrategy
{
public bool IsTransient(Exception ex)
{
return true;
}
}
As for the retry strategy, TOPAZ offers three options:
FixedInterval
Incremental
ExponentialBackoff
For example, here's the TOPAZ equivalent of what the Azure Client Storage Library uses for default:
int retries = 3;
var minBackoff = TimeSpan.FromSeconds(3.0);
var maxBackoff = TimeSpan.FromSeconds(120.0);
var deltaBackoff= TimeSpan.FromSeconds(4.0);
var strategy = new ExponentialBackoff(retries, minBackoff, maxBackoff, deltaBackoff);
For more information see http://msdn.microsoft.com/en-us/library/hh680901(v=pandp.50).aspx
EDIT Note that if your request contains an HttpContent object, you'll have to regenerate it every time as that will be disposed by HttpClient as well (thanks for catching that Alexandre Pepin). For example () => httpClient.PostAsync(url, new StringContent("foo"))).
Duplicating the StringContent isn't probably the best idea. But simple modification could fix the problem. Just modify the function and create the StringContent object inside of the loop, something like:
public HttpResponseMessage ExecuteWithRetry(string url, string contentString)
{
HttpResponseMessage result = null;
bool success = false;
using (var client = new HttpClient())
{
do
{
result = client.PostAsync(url, new StringContent(contentString)).Result;
success = result.IsSuccessStatusCode;
}
while (!success);
}
return result;
}
and then call it
ExecuteWithRetry("http://www.requestb.in/xfxcva" /*valid url*/, "Hello World");
This is what I achieved using polly.
nuget
https://www.nuget.org/packages/Microsoft.Extensions.Http.Polly
https://www.nuget.org/packages/Polly
using Polly;
using Polly.Extensions.Http;
//// inside configure service
services.AddHttpClient("RetryHttpClient", c =>
{
c.BaseAddress = new Uri($"{configuration["ExternalApis:MyApi"]}/");
c.DefaultRequestHeaders.Add("Accept", "application/json");
c.Timeout = TimeSpan.FromMinutes(5);
c.DefaultRequestHeaders.ConnectionClose = true;
}).AddPolicyHandler(GetRetryPolicy());
//// add this method to give retry policy
private static IAsyncPolicy<HttpResponseMessage> GetRetryPolicy()
{
return HttpPolicyExtensions
//// 408,5xx
.HandleTransientHttpError()
//// 404
.OrResult(msg => msg.StatusCode == HttpStatusCode.NotFound)
//// 401
.OrResult(msg => msg.StatusCode == HttpStatusCode.Unauthorized)
//// Retry 3 times, with wait 1,2 and 4 seconds.
.WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)));
}
This builds off the accepted answer but adds the ability to pass in the amount of retries, plus the ability to add non-blocking delays / wait time to each request. It also uses a try catch to ensure the retry continues to happen after an exception has occurred. And last, I added code to break out of the loop in the case of BadRequests, you don't want to resend the same bad request multiple times.
public class HttpRetryHandler : DelegatingHandler
{
private int MaxRetries;
private int WaitTime;
public HttpRetryHandler(HttpMessageHandler innerHandler, int maxRetries = 3, int waitSeconds = 0)
: base(innerHandler)
{
MaxRetries = maxRetries;
WaitTime = waitSeconds * 1000;
}
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
HttpResponseMessage response = null;
for (int i = 0; i < MaxRetries; i++)
{
try
{
response = await base.SendAsync(request, cancellationToken);
if (response.IsSuccessStatusCode)
{
return response;
}
else if(response.StatusCode == HttpStatusCode.BadRequest)
{
// Don't reattempt a bad request
break;
}
}
catch
{
// Ignore Error As We Will Attempt Again
}
finally
{
response.Dispose();
}
if(WaitTime > 0)
{
await Task.Delay(WaitTime);
}
}
return response;
}
}
With RestEase And Task, on retry with httpClient reused in many call (singleton) it frezze and throw TaskCanceledException.
To fix this whe need to Dispose() the failed response before retry
public class RetryHandler : DelegatingHandler
{
// Strongly consider limiting the number of retries - "retry forever" is
// probably not the most user friendly way you could respond to "the
// network cable got pulled out."
private const int MaxRetries = 3;
public RetryHandler(HttpMessageHandler innerHandler)
: base(innerHandler)
{ }
protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
HttpResponseMessage response = null;
for (int i = 0; i < MaxRetries; i++)
{
response = await base.SendAsync(request, cancellationToken).ConfigureAwait(false);
if (response.IsSuccessStatusCode) {
return response;
}
response.Dispose();
}
return response;
}
}
I tried it and worked while using unit and integration tests. However, it stuck when I actually called from REST URL. I found this interesting post which explains why it gets stuck at this line.
response = await base.SendAsync(request, cancellationToken);
The fix to this is that you have .ConfigureAwait(false) added at the end.
response = await base.SendAsync(request, token).ConfigureAwait(false);
I also added create linked token part there like this.
var linkedToken = cancellationToken.CreateLinkedSource();
linkedToken.CancelAfter(new TimeSpan(0, 0, 5, 0));
var token = linkedToken.Token;
HttpResponseMessage response = null;
for (int i = 0; i < MaxRetries; i++)
{
response = await base.SendAsync(request, token).ConfigureAwait(false);
if (response.IsSuccessStatusCode)
{
return response;
}
}
return response;
You also refer to Building a Transient Retry Handler for the .NET HttpClient.
Visit refer to KARTHIKEYAN VIJAYAKUMAR post.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Data.SqlClient;
using System.Net.Http;
using System.Threading;
using System.Diagnostics;
using System.Net;
using Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling;
namespace HttpClientRetyDemo
{
class Program
{
static void Main(string[] args)
{
var url = "http://RestfulUrl";
var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, url);
var handler = new RetryDelegatingHandler
{
UseDefaultCredentials = true,
PreAuthenticate = true,
Proxy = null
};
HttpClient client = new HttpClient(handler);
var result = client.SendAsync(httpRequestMessage).Result.Content
.ReadAsStringAsync().Result;
Console.WriteLine(result.ToString());
Console.ReadKey();
}
}
/// <summary>
/// Retry Policy = Error Detection Strategy + Retry Strategy
/// </summary>
public static class CustomRetryPolicy
{
public static RetryPolicy MakeHttpRetryPolicy()
{
// The transient fault application block provides three retry policies
// that you can use. These are:
return new RetryPolicy(strategy, exponentialBackoff);
}
}
/// <summary>
/// This class is responsible for deciding whether the response was an intermittent
/// transient error or not.
/// </summary>
public class HttpTransientErrorDetectionStrategy : ITransientErrorDetectionStrategy
{
public bool IsTransient(Exception ex)
{
if (ex != null)
{
HttpRequestExceptionWithStatus httpException;
if ((httpException = ex as HttpRequestExceptionWithStatus) != null)
{
if (httpException.StatusCode == HttpStatusCode.ServiceUnavailable)
{
return true;
}
else if (httpException.StatusCode == HttpStatusCode.MethodNotAllowed)
{
return true;
}
return false;
}
}
return false;
}
}
/// <summary>
/// The retry handler logic is implementing within a Delegating Handler. This has a
/// number of advantages.
/// An instance of the HttpClient can be initialized with a delegating handler making
/// it super easy to add into the request pipeline.
/// It also allows you to apply your own custom logic before the HttpClient sends the
/// request, and after it receives the response.
/// Therefore it provides a perfect mechanism to wrap requests made by the HttpClient
/// with our own custom retry logic.
/// </summary>
class RetryDelegatingHandler : HttpClientHandler
{
public RetryPolicy retryPolicy { get; set; }
public RetryDelegatingHandler()
: base()
{
retryPolicy = CustomRetryPolicy.MakeHttpRetryPolicy();
}
protected async override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
HttpResponseMessage responseMessage = null;
var currentRetryCount = 0;
//On Retry => increments the retry count
retryPolicy.Retrying += (sender, args) =>
{
currentRetryCount = args.CurrentRetryCount;
};
try
{
await retryPolicy.ExecuteAsync(async () =>
{
responseMessage = await base.SendAsync(request, cancellationToken)
.ConfigureAwait(false);
if ((int)responseMessage.StatusCode > 500)
{
// When it fails after the retries, it would throw the exception
throw new HttpRequestExceptionWithStatus(
string.Format("Response status code {0} indicates server error",
(int)responseMessage.StatusCode))
{
StatusCode = responseMessage.StatusCode,
CurrentRetryCount = currentRetryCount
};
}// returns the response to the main method(from the anonymous method)
return responseMessage;
}, cancellationToken).ConfigureAwait(false);
return responseMessage;// returns from the main method => SendAsync
}
catch (HttpRequestExceptionWithStatus exception)
{
if (exception.CurrentRetryCount >= 3)
{
//write to log
}
if (responseMessage != null)
{
return responseMessage;
}
throw;
}
catch (Exception)
{
if (responseMessage != null)
{
return responseMessage;
}
throw;
}
}
}
/// <summary>
/// Custom HttpRequestException to allow include additional properties on my exception,
/// which can be used to help determine whether the exception is a transient
/// error or not.
/// </summary>
public class HttpRequestExceptionWithStatus : HttpRequestException
{
public HttpStatusCode StatusCode { get; set; }
public int CurrentRetryCount { get; set; }
public HttpRequestExceptionWithStatus()
: base() { }
public HttpRequestExceptionWithStatus(string message)
: base(message) { }
public HttpRequestExceptionWithStatus(string message, Exception inner)
: base(message, inner) { }
}
}
i have almost the same issue.
HttpWebRequest queueing library, which guarantees request delivery
I just updated (see EDIT3) my approach to avoid crashes, but i still need general mechanism to guarantee message delivery (or re-delivery in case message was not delivered).
I have the same problem and solved . it's about the "StringContent" / "HttpContent"
Please check Amogh Natu's Blog which help me to solve this problem
The problem with this code is, when the first call to PostAsync is
made and it fails, the httpContent object is disposed. This is as
designed in the HttpClient class. Refer the comment in this method.
Although this seems odd, they intent to do this so that the user
doesn’t have to do this explicitly and also to avoid the same request
being Posted more than once.
So what happens is, when the first call fails, httpContent is
disposed, then as we have retry mechanism, it tries to make the post
call again, now with a disposed object and hence this time, the call
fails with an ObjectDisposedException.
An easy way to resolve this is to NOT use a variable to store
httpContent and instead, create http content directly while making the
call. Something like this.
http://amoghnatu.net/2017/01/12/cannot-access-a-disposed-object-system-net-http-stringcontent-while-having-retry-logic/
Adding an answer that uses both Polly + Retry policy + per-retry Timeout policy, as top answer does not address that:
Policy
.Handle<HttpRequestException>()
.Or<TaskCanceledException>()
.Or<TimeoutRejectedException>()
.OrResult<HttpResponseMessage>(x => !x.IsSuccessStatusCode)
.WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(3, retryAttempt)))
.WrapAsync(
Policy.TimeoutAsync(TimeSpan.FromSeconds(1), delegate (Context ctx, TimeSpan timeSpan, Task task)
{
// Do some on-timeout action
return Task.CompletedTask;
})
)
.ExecuteAsync(() =>
{
return httpclient.PostAsync(url, httpRequest);
});
//Could retry say 5 times
HttpResponseMessage response;
int numberOfRetry = 0;
using (var httpClient = new HttpClient())
{
do
{
response = await httpClient.PostAsync(uri, content);
numberOfRetry++;
} while (response.IsSuccessStatusCode == false | numberOfRetry < 5);
}
return response;
.........