I am using a static HttpClient (for scalability reasons - see What is the overhead of creating a new HttpClient per call in a WebAPI client?) and would like to be able to cancel individual requests that take too long. There is an overload on SendAsync that takes a CancellationToken - but I don't know if it's thread-safe since my HttpClient instance is static. For example, if I have several requests being sent thru the HttpClient simultaneously and I try to cancel one, does it cancel the right one?
I looked thru the HttpClient code and at first glance it doesn’t look like it is thread-safe to do so since the cancellation is sent to the HttpClientHandler (which is the same for all requests). But I could be missing something. So my questions are:
Can I cancel individual requests on a static HttpClient?
If not, how can I accomplish this?
NOTE:
Since testing this, requires a way to reliably create a race condition, in code that I do not control, I don't see a way to test this.
Each SendAsync call is totally independent from each other, canceling the token for one request does not cancel other outstanding requests.
Your assumption that because HttpClientHandler is shared for all requests that means all requests get canceled is incorrect. If you look in to the decompiled source of HttpClientHandler you will see
[__DynamicallyInvokable]
protected internal override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
if (request == null)
throw new ArgumentNullException(nameof (request), SR.net_http_handler_norequest);
this.CheckDisposed();
if (Logging.On)
Logging.Enter(Logging.Http, (object) this, nameof (SendAsync), (object) request);
this.SetOperationStarted();
TaskCompletionSource<HttpResponseMessage> completionSource = new TaskCompletionSource<HttpResponseMessage>();
HttpClientHandler.RequestState state = new HttpClientHandler.RequestState();
state.tcs = completionSource;
state.cancellationToken = cancellationToken;
state.requestMessage = request;
try
{
HttpWebRequest prepareWebRequest = this.CreateAndPrepareWebRequest(request);
state.webRequest = prepareWebRequest;
cancellationToken.Register(HttpClientHandler.onCancel, (object) prepareWebRequest);
if (ExecutionContext.IsFlowSuppressed())
{
IWebProxy webProxy = (IWebProxy) null;
if (this.useProxy)
webProxy = this.proxy ?? WebRequest.DefaultWebProxy;
if (this.UseDefaultCredentials || this.Credentials != null || webProxy != null && webProxy.Credentials != null)
this.SafeCaptureIdenity(state);
}
Task.Factory.StartNew(this.startRequest, (object) state);
}
catch (Exception ex)
{
this.HandleAsyncException(state, ex);
}
if (Logging.On)
Logging.Exit(Logging.Http, (object) this, nameof (SendAsync), (object) completionSource.Task);
return completionSource.Task;
}
The cancellation token is getting wrapped up in a new HttpClientHandler.RequestState state object every call of SendAsnyc, when the token is canceled only the state.webRequest associated with that state object is the one that will be canceled.
Just got confirmation from the Product Team at Microsoft:
Yes, it is completely safe to cancel an individual request using the
cancellation token passed into the various HttpClient.SendAsync,
.GetAsync, etc. methods. It does not matter that the HttpClient is
"static". The cancellation token passed into the method is used for
that particular request only.
Related
An HttpRequestMessage object can only be used one time; future attempts to use the same object throw an exception. I'm using Polly to retry some requests and I'm hitting this issue. I know how I can clone a request, there are plenty of examples on SO, but I can't figure out how to clone a request and send that new request whenever Polly retries. How can I accomplish this?
These are my policies, for reference. This is a Xamarin app. I want to retry a few times in case of network failures, and if the response is unauthorized I want to re-auth with saved credentials and try the original request again.
public static PolicyWrap<HttpResponseMessage> RetryPolicy
{
get => WaitAndRetryPolicy.WrapAsync(ReAuthPolicy);
}
private static IAsyncPolicy WaitAndRetryPolicy
{
get => Policy.Handle<WebException>().WaitAndRetryAsync(4, _ => TimeSpan.FromSeconds(2));
}
private static IAsyncPolicy<HttpResponseMessage> ReAuthPolicy
{
get => Policy.HandleResult<HttpResponseMessage>(x => x.StatusCode == HttpStatusCode.Unauthorized)
.RetryAsync((_, __) => CoreService.LogInWithSavedCredsAsync(true));
}
This doesn't work because of the HttpRequestMessage reuse, but it's what I'm trying to accomplish:
var request = new HttpRequestMessage(HttpMethod.Post, "some_endpoint")
{
Content = new StringContent("some content")
};
request.Content.Headers.ContentType = new MediaTypeHeaderValue("application/json");
var policyResponse = await ConnectivityHelper.RetryPolicy
.ExecuteAndCaptureAsync(() => _client.SendAsync(request)).ConfigureAwait(false);
// handle outcome
The code to throw InvalidOperationException if an HttpRequestMessage is reused is a validation step within HttpClient itself.
Source code link
private static void CheckRequestMessage(HttpRequestMessage request)
{
if (!request.MarkAsSent())
{
throw new InvalidOperationException(SR.net_http_client_request_already_sent);
}
}
Source code link
internal bool MarkAsSent()
{
return Interlocked.Exchange(ref sendStatus, messageAlreadySent) == messageNotYetSent;
}
You can put the polly retry policy in a DelegatingHandler and that works. It also provides a nice SoC (separation of concerns). If, in future, you want to not retry or change retry behavior, you simply remove the DelegatingHandler or change it. Note to dispose off the HttpRequestMessage and intermediate HttpResponseMessages objects. Here is one that I use with good results (retry policy).
Your question is an open-ended, and generally SO is not good for those (see). But here goes. I call this a "reactive" approach as it uses the token right up until its ttl, and fetches the new one. Note that this doesn't incur 401s by using the token ttl.
# gets token with its ttl
tokenService: iTokenService
# use retry policy in DH here
httpClient
string getTokenAsync():
# calls out for token
# note: tokens typically have a ttl
# returns cached token till its tll, or gets a new token which is then cached
cachedTokenService: iCachedTokenService
tokenCached
tokenTtl
iTokenService
string getTokenAsync():
# returns tokenCached or gets a new token based on ttl
# note: fetches with some buffer before ttl to avoid failures on edge
# note: buffer as 2x http timeout is good enough
# DH that adds the cached token to the outgoing "work" request
tokenHandler: delegatingHandler
iCachedTokenService
task<response> sendAsync(request, ct):
# gets token, and adds token to request header
# worker service
workService: iWorkService
# uses tokenHandler DH
httpClient
workAsync():
# ...
Well, the simplest solution is to move the creation of the HttpRequestMessage inside the ExecuteAndCaptureAsync delegate. In other words do not reuse rather recreate it:
var policyResponse = await ConnectivityHelper.RetryPolicy
.ExecuteAndCaptureAsync(async () => {
var request = new HttpRequestMessage(HttpMethod.Post, "some_endpoint")
{
Content = new StringContent("some content", Encoding.UT8, "application/json")
};
return await _client.SendAsync(request)).ConfigureAwait(false);
});
Or simply prefer PostAsync over SendAsync
var policyResponse = await ConnectivityHelper.RetryPolicy.ExecuteAndCaptureAsync(
async () =>
await _client.PostAsync("some_endpoint",
new StringContent("some content", Encoding.UT8, "application/json"))
.ConfigureAwait(false)
});
I have an OWIN-based ASP.NET Web API hosted in a Windows Service. Most of my ApiController actions are async, and accept CancellationToken parameters:
[Route("data/{id}")]
public async Task<IHttpActionResult> GetSomeDataAsync(int id, CancellationToken token)
{
try
{
using (var _dataSource = ...)
{
return Ok(await _dataSource.GetDataAsync(id, token));
}
}
catch (OperationCanceledException ex)
{
return StatusCode(HttpStatusCode.NoContent);
}
}
Using the built-in request-cancellation features of Web API, if the client cancels the request, token is signaled and _dataSource handles it appropriately and throws the OperationCanceledException.
So far, so great.
But when my host process terminates (that is, the Windows Service stops), token isn't signaled and the cancellation-and-bail-out process isn't graceful.
I'm aware of the OWIN environment dictionary's host.onAppDisposing property, and I've dug into the source for the Microsoft.Owin[.*] and Microsoft.AspNet.WebApi.* packages to try and figure out where GetSomeDataAsync's token argument is coming from, but I'm not sure how to connect the pieces together.
I'd like to do something like
class WebServiceInAWindowsService : ServiceBase
{
private readonly CancellationTokenSource _cts = new CancellationTokenSource();
...
protected override void OnStop()
{
_cts.Cancel();
}
}
But I'm not sure how to get _cts to be the source of the CancellationTokens that get fed to my actions, while not breaking the request-cancellation feature that's working well.
I'm thinking that CancellationTokenSource.CreateLinkedTokenSource() might be useful, but I'm not seeing how to put the pieces together.
Can you help? Thanks!
host.onAppDisposing is triggered when you call Dispose on the value returned from WebApp.Start.
https://github.com/aspnet/AspNetKatana/blob/9f6e09af6bf203744feb5347121fe25f6eec06d8/src/Microsoft.Owin.Hosting/Engine/HostingEngine.cs#L302-L308
https://github.com/aspnet/AspNetKatana/blob/9f6e09af6bf203744feb5347121fe25f6eec06d8/src/Microsoft.Owin.Hosting/Engine/HostingEngine.cs#L112
GetSomeDataAsync's is only associated with the request disconnect token by default (e.g. owin.CallCancelled). Via middleware or otherwise you can replace it with a linked TCS that's also connected to host.onAppDisposing.
Something like:
app.Use(async (env, next) =>
{
var reqAbt = env.Get<CancellationToken>("owin.CallCancelled");
var appAbt = env.Get<CancellationToken>("host.onAppDisposing");
using (linked = CancellationTokenSource.CreateLinkedTokenSource(reqAbt, appAbt))
{
env["owin.CallCancelled"] = linked.Token;
await next();
env["owin.CallCancelled"] = reqAbt;
}
});
We have a three tier infrastructure (front end which is all Web API 2, Middleware which accepts API calls from front end and runs business logic and databases access, then the DB)
I'm trying to find out why our app locks up when I take the middle tier down. We use Memcached for all the reads and the front end serves the cached data just fine, but one of the calls that is made checks to see if the user is logged in. Running on my local machine with one app pool, that call locks the thread (I think) and prevents the rest of the calls from doing anything until the timeout on the autologin call expires.
The code path looks like this:
call to api/autologin --> front end API calls Client.SendAsync (our custom method for passing along data to the middleware), this tries to call the middlewware by using HttpClient.SendAsAsync with a timeout of 3 minutes (Probably should shorten this)
My expectation is that this should release this thread while we are waiting. That does not appear to be the result.
The REALLY weird thing is that when the middleware is down the Client.SendAsync gets ran MANY time, like 10. I thought this was maybe HTTP 2.0 in Chrome, but I switched to Fiddler and it did the same thing. Very weird.
So, two questions.
1. What's with the multiple calls?
2. Why do the threads appear to be getting locked?
Here's the code.
/// <summary>
/// Auto login user if they have the persistent cookies.
/// </summary>
/// <returns>The groups the logged in user has access to in the form of a
LoggedInUserData object.</returns>
[Route("api/cms/autologin/")]
[HttpGet]
public async Task<HttpResponseMessage> AutoLogin()
{
HttpResponseMessage response = await Client.SendAsync(this.Request);
return this.LoginCacheHelper(response);
}
That calls
public static async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request)
{
return await Client.SendAsync<string>(request, null, null, false);
}
Which calls
public static async Task<HttpResponseMessage> SendAsync<T>(HttpRequestMessage request, T content = null, string route = null, bool isFile = false, TimeSpan? timeout = null) where T : class
{
// Validate all internal certs.
ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
// Determine the route and make sure route has a starting forward slash.
if (!string.IsNullOrWhiteSpace(route) && route.StartsWith("http"))
{
// Check to make sure this is a selinc.com domain for security purposes.
if (Sel.Utils.Validation.UriValidation.IsSelincDomain(route))
{
request.RequestUri = new Uri(route);
}
else
{
return new HttpResponseMessage(HttpStatusCode.BadRequest);
}
}
else
{
string middlewareRoute = GetRoute(route, request);
// Change Uri to middle ware.
request.RequestUri = new Uri(Config.MwareSiteUrl + middlewareRoute);
}
// Remove host header
request.Headers.Host = string.Empty;
// Set content of request.
// File content will be kept on the request as is.
if (content != null && !isFile)
{
request.Content = new ObjectContent<T>(content, new JsonMediaTypeFormatter());
}
else if (!isFile)
{
request.Content = null;
}
// Client handler set use cookies to false which will pass along the current cookies
HttpClientHandler clientHandler = new HttpClientHandler() { UseCookies = false };
// The HttpClient object
HttpClient client = new HttpClient(clientHandler);
client.Timeout = timeout ?? new TimeSpan(0, 3, 0);
// Send the request
return await client.SendAsync(request);
}
Adding image of the Network log in Chrome to illustrate the behavior.
Note that if I remove the API call to the autologin, everything works fine. It's the only call in this stack that hits the back end.
Also note: If I modify the SendAsync method to just return a new HttpResponseMessage (and thus do no work) then the autologin basically does nothing, returns quickly and site loads as it should, with the middleware server down. This is just to prove that it is the autologin API call causing the problem. The autologin API call is the only method calling SendAsync at this time so it's a valid test.
// Send the request
////return await client.SendAsync(request);
return new HttpResponseMessage();
I'm running into an issue with the .NET HttpClient class (.NET 4.5.1, System.Net.Http v4.0.0.0). I'm calling HttpClient.GetAsync, passing in a CancellationToken (as part of a Nuget package that abstracts calls between webservices). If the token has been cancelled before the call is made, the request goes through without throwing an exception. This behavior doesn't seem correct.
My test (incomplete, not fully written - no exception check):
[TestMethod]
public async Task Should_Cancel_If_Cancellation_Token_Called()
{
var endpoint = "nonexistent";
var cancellationTokenSource = new CancellationTokenSource();
var _mockHttpMessageHandler = new MockHttpMessageHandler();
_mockHttpMessageHandler
.When("*")
.Respond(HttpStatusCode.OK);
var _apiClient = new ApiClientService(new HttpClient(_mockHttpMessageHandler));
cancellationTokenSource.Cancel();
var result = await _apiClient.Get<string>(endpoint, null, cancellationTokenSource.Token);
}
The method I'm testing:
public async Task<T> Get<T>(string endpoint, IEnumerable<KeyValuePair<string, string>> parameters = null, CancellationToken cancellationToken = default(CancellationToken))
{
var builder = new UriBuilder(Properties.Settings.Default.MyEndpointHost + endpoint);
builder.Query = buildQueryStringFromParameters(parameters);
_httpClient.DefaultRequestHeaders.Accept.Clear();
_httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
try
{
// After this, we really shouldn't continue.
var request = await _httpClient.GetAsync(builder.Uri, cancellationToken);
if (!request.IsSuccessStatusCode)
{
if (request.StatusCode >= HttpStatusCode.BadRequest && request.StatusCode < HttpStatusCode.InternalServerError)
{
throw new EndpointClientException("Service responded with an error message.", request.StatusCode, request.ReasonPhrase);
}
if (request.StatusCode >= HttpStatusCode.InternalServerError && (int)request.StatusCode < 600)
{
throw new EndpointServerException("An error occurred in the Service endpoint.", request.StatusCode, request.ReasonPhrase);
}
}
var json = await request.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<T>(json);
}
catch (Exception ex)
{
throw;
}
}
I know that I can check the status of the cancellation token before calling HttpClient.GetAsync and throw if cancellation has been requested. I know that I can also register a delegate to cancel the HttpClient request. However, it seems as though passing the token to the HttpClient method should take care of this for me (or, else, what's the point?) so I'm wondering if I'm missing something. I don't have access to the HttpClient source code.
Why is HttpClient.GetAsync not checking my cancellation token and aborting its process when I pass it in?
HttpClient doesn't check the cancellation token itself, it passes it on to the message handler when it calls its SendAsync method. It then registers to the continuation on the task returned from SendAsync and will set its own task as cancelled if the task returned from the message handler was cancelled.
So the problem in your scenario is in your implementation of MockHttpMessageHandler which seems doesn't check the cancellation token.
Note, that if HttpClient is called via its empty constructor, it internally uses HttpClientHandler which registers a delegate on the cancellation token that aborts the request and cancels the task.
Could someone point me to a resource that would help explain how web api (specifically using Owin Self Host) handles request cancellation?
Here's the sequence of events i'm observing:
someone makes a GET from chrome
Api controller (through some layers) fires off an async SQL query
someone hits the X button in chrome (i don't know exactly what happens on socket for this)
What happens next in Web Api??
There was some code running in a controller, does the thread running it get aborted? If it was an async controller awaiting another Task, does that task still have an awaiter in case it returns with an exception?
For context: I do have an async controller awaiting a Task (this is the only call site) which looks to be throwing an unobserved exception in some edge cases. I haven't been able to isolate or re-produce yet :)
I did find something called HttpResponse.ClientDisconnectedToken, but don't know well that is supported in Owin Selfhost + is it even the good thing to use for all user cancels.
I've dealt with this by handing the System.OperationCanceledException in a custom middleware I've registered before WebApi.
public class ExceptionHanldingMiddleware : OwinMiddleware
{
public override async Task Invoke(IOwinContext context)
{
try
{
await Next.Invoke(context);
}
catch (OperationCanceledException) when (context.Request.CallCancelled.IsCancellationRequested)
{
//swallow user-agent cancelling request.
_log.Trace($"client disconnected on request for: {context.Request.Path}.");
}
catch (Exception ex)
{
_log.Error(ex);
context.Response.StatusCode = (int) HttpStatusCode.InternalServerError;
context.Response.ReasonPhrase = "Internal Server Error";
}
}
}
As you stated that your async controller is awaiting for a Task, which sometimes got some exception, I suggest you ContinueWith extension method for a task, which can be run only then your task is faulted, like this:
task.ContinueWith(
t =>
logger.Error(t.Exception.Message, t.Exception);
, TaskContinuationOptions.OnlyOnFaulted);
This is a default mechanism to handle the exceptions, and this will work in OWIN application.
Second, as for the cancellation: task can be started with a CancellationToken structure, which can be used for a cancelling the task during the execution. You can read more in the MSDN article.
HttpResponse.ClientDisconnectedToken is used for a situation when the client has been disconnected and the request should not be proceed in execution.
You can use this token, or create your own with CancellationTokenSource, like this:
var source = new CancellationTokenSource();
var token = source.Token;
var task = Task.Factory.StartNew(() =>
{
// Were we already canceled?
ct.ThrowIfCancellationRequested();
var moreToDo = true;
while (moreToDo)
{
// Poll on this property if you have to do
// other cleanup before throwing.
if (ct.IsCancellationRequested)
{
// Clean up here, then...
ct.ThrowIfCancellationRequested();
}
}
}, token);