I need to write code to handle the 2 distinct types of failures that can arise after issuing an HTTP request using an HttpClient object that comes with .Net Framework 4.5:
Connectivity failures, when the server can not be reached due to connection problems - e.g. no connection available, timeout expiration, incorrect network settings etc.
Although technically not a failure, the case when the request successfully reaches the server but the server responds with an HTTP error status code (e.g. 404, 400, 409, 500 etc.)
It is very important that one failure type is not treated as the other. Which is the correct and reliable way to do this? The EnsureSuccessStatusCode() method seems to help with at least the second case, but the documentation is disappointingly scarce.
After some manual testing, the good news is that there seems to be a reliable method to separate the 2 types of errors, but the bad news is that it's not consistently the same on all platforms - at least not on Windows Phone 8. On .Net Framework 4.5 Full and on Windows Store App types of project, the following code does the trick:
try
{
var response = await httpClient.PostAsync(uri, content);
if (!response.IsSuccessStatusCode)
{
// handle the second type of error (404, 400, etc.)
}
}
catch (HttpRequestException ex)
{
// handle the first type of error (no connectivity, etc)
}
On Windows Phone 8, unfortunately, both types are handled under the if condition: IsSuccessStatusCode is set to false and no HttpRequestException (or any other exception, for that matter) is raised. For WP8 I'm using the portable class library implementation of HttpClient (http://blogs.msdn.com/b/bclteam/archive/2013/02/18/portable-httpclient-for-net-framework-and-windows-phone.aspx), I asked them if this inconsistency can be fixed, hopefully it can and will soon be.
Related
I'm working with a web API that will return code 404 if querying a data that doesn't exist, or other errors if the data is malformed of there's some other problem. Which then results in an HttpRequestException.
Now I'm thinking about a detail. I'm using Polly on that HttpClient connection to ensure it retries in case of communication problems.
In this case, will it work as expected, or will Polly keep retrying in the case of server-thrown errors like "not found" or "bad request"?
I'm configuring it like this
services.AddHttpClient<OntraportHttpClient>()
.AddTransientHttpErrorPolicy(p =>
p.WaitAndRetryAsync(3, _ => TimeSpan.FromMilliseconds(600)));
You have a bit of misunderstanding, 400 Bad Request or 404 Not Found will not result in HttpRequestException.
Unless you call EnsureSuccessStatusCode explicitly.
AddTransientHttpErrorPolicy will check the followings:
408 Timeout
5xx Server error
HttpRequestException
So as you can see neither 400, 404, nor 429 Too Many Requests (typical response code in case of back-pressure) will cause your Polly policy to be triggered. Unless you explicitly call EnsureSuccessStatusCode method.
UPDATE: Adding DELETE use case
Use Case
Let's suppose we have a REST service which exposes a removal functionality of a given resource (addressed by a particular URL and via the DELETE HTTP verb).
This removal can end up in one of the 3 different states from consumption point of view:
Succeeded
Already Done
Failed
You can find several arguments on the internet which is the correct state for succeeded. It can either 200 (OK) with body or 204 (No Content) without body or 202 (Accepted) if it is asynchronous. Sometimes 404 (Not Found) is also used.
The already done state can occur when you try to delete an already deleted item. Without soft deletion it is hard to tell that the given resource has ever existed before or it was never been part of your system. If you have soft deletion, then the service could return 404 for an already deleted resource and 400 (Bad Request) for an unknown resource.
Whenever something fails during the request processing then it can be treated as temporary or permanent failure. If there is a network issue then it can be considered as a temporary/transient issue (this can be manifested as HttpRequestException). If there is a database outage and the service is able to detect it then it can fail fast and return with a 5XX response or it can try to fail over. If there are too many pending requests then the service may consider to throttle them and use back-pressure to shed the load. It might return with 429 (Too Many Requests) along with the appropriate Retry-After header.
Permanent errors, like service has been shut down forever or active refusal of network connection attempts under TLS 1.3 need human intervention to fix them.
Idempotency
Whenever we are talking about retry pattern we need to consider the followings:
The potentially introduced observable impact is acceptable
The operation can be redone without any irreversible side effect
The introduced complexity is negligible compared to the promised reliability
The second criteria is usually referred as Idempotency. It says that if you call the method / endpoint multiple times with the same input then it should return the same output without any side effect.
If your service's removal functionality can be considered as idempotent then there is no such state as Already done. If you call it 100 times then it should always return with "yepp, that's gone". So with this is mind it might make sense to return with either 204 or 404 in case of idempotent deletion.
Resilient strategy
Whenever we are talking about strategy it means for me a chain of resilient policies. If a former policy could not "fix" the problem then the latter would try to do that (so there is a policy escalation).
Server-side: You can use Bulk-head policy to have control over the maximum number of concurrent calls but if the threshold has been exceeded then you can start to throttle requests.
Client-side: You can have a timeout for each individual request and you can apply retry policy in case of temporary/transient failure. You can also define a global timeout for all your retry attempts. Or you can apply a circuit breaker to monitor successive failures and back-off for a given period of time if the service is treated as overwhelmed or malfunctioning.
My 2 cents is applying a single resilient policy on the client-side might not be enough to have a robust and resilient system. It might require several policies (on both sides) to establish a communication protocol for problematic periods.
I have a strange issue I'm trying to triage having to do with the new HttpClient on .NET Core 2.1. From this article here (https://blogs.msdn.microsoft.com/dotnet/2018/04/11/announcing-net-core-2-1-preview-2/) I know that the HttpClient has been completely re-written to use a different low level library for handling HTTP requests. I'm wondering if anyone has seen any issues with the new implementation.
What I'm seeing is a strange case where my application (.NET Core 2.1) which sends a POST request to some API periodically (every 10 seconds) a few of times every 15 min it will throw an exception with the error: An error occurred while sending the request.; The server returned an invalid or unrecognized response.
No other details are available, it's just an exception when I make a call like this:
using (var res = await _httpClient.PostAsync(uriBuilder.Uri, new StringContent(serializedRequestBody, Encoding.UTF8, "application/json")))
{
//Do something here
}
The exception caught is a System.Net.Http.HttpRequestException and it has some inner exception with the above error message.
So as I mentioned this does NOT happen all time, it happens seemingly at random, or at least I can not discern any particular pattern. All I can say is these POST requests are made once every 10 seconds 24/7 and anywhere between 5% and 10% of the POST requests fail with the above exception.
So used tcdump and piped it into wireshark to examine the requests to see what's actually happening when the requests fail and what i see is the following:
On a good POST I see: my app sends the request to server, server sends response back, my app sends ACK to server and server responds with FIN,ACK. Done. Good Stuff.
On POST which gets the above exception I see the following: my app sends the request to server, and almost immediately after (like a few milliseconds after) my application sends FIN, ACK to server.
This seems consistent with what I see in my application logs, which show that the request duration is 0 before the exception is thrown.
So what it looks like to me is, my application sends the request and then immediately after closes the connection for some reason. However, I don't understand why this happens. I tried comparing the raw HTTP requests (good POST vs bad POST) to see any differences and I can not see any difference.
One last thing to mention, is that I ONLY see this in applications running on .NET Core 2.1. When I run my application on .NET 2.0 I do not see this problem. Also when I use the same library (where the HTTP call is being made) in the .NET 4.5.1 application (I use multi-targeting to compile the library targeting .net standard and net451) I also do NOT see this problem. So it seems to affect only .NET Core 2.1
Any ideas of where I can go from here? Is there something else I should look for ? How would someone go about trying to triage this type of issue ?
[EDIT] I added a screenshot of the wireshark output which shows the last POST request the server never does not respond before the client sends FIN,ACK
[EDIT]
#Svek Pointed out something in the comments about the sequence of ACKs. I think there maybe something here, because (in the screenshot) after the very last POST there is a FIN, ACK and it shows Ack=7187, so I look back I see the previous FIN,ACK had sequence=7186. Now, I'm by far not an expect in TCP or networking so I maybe saying something completely dumb, but does that mean that the last FIN,ACK (which comes from my host to server) is essentially my host FIN,ACK'ing the previous FIN,ACK (from server to my host) and essentially closing the connection.
So since the next POST is made to the same host:port, using the same connection and yet the connection is closed (via that last FIN,ACK) that's why I never get a response back?
When I tested my web-app based on MVC5 I found that HttpRequestValidationException returns 500 from GetHttpCode().
I catch this exception during security testing of requests to the server. MVC performs request validation and throws this exception, I processed it in the Application_Error and got 500 code returned, but, in fact, if user passed wrong input to server I should return 400 error (bad request). so don't understand why the exception has 500 error. Where I am wrong ?
There are four reasons why 500 might be considered more appropriate.
One is that it isn't clear when a HttpRequestValidationException happens who did the wrong thing. It could be that the user quite clearly sent something incorrect, but it's also possible that the server incorrectly rejected something that was perfectly fine. Considering that this is the default behaviour on many requests that could be malicious, it's probable that the majority of cases where its thrown in response to non-malicious requests (the user isn't actively trying to do the site harm) fall into this category. Hence it could either be something that should be consider 4xx or 5xx and 5xx is a good assumption (don't blame the client when you aren't sure it's the client's fault).
When it happens in a non-malicious case it generally needs attention from a developer, which is only sometimes true of the 4xx codes (a 404 might need developer-attention because the 404 was caused by a bad link on the same site, but it might not). The sort of feedback one gets from 500 responses is more appropriate.
The general model ASP.NET started with pretty much mapped all exceptions to 500 responses, which does make sense a lot of the time. MVC has (sensibly) replaced that with more exceptions that map to other error conditions, but HttpRequestValidationException predates MVC.
Now those points made, I can't say I agree with any of them. I'd still say that HttpRequestValidationException means a client error and if it was mis-identified as such, well it was still identified as such, so it's clearly a 400.
However, the final reason is:
While the description of 400 status in RFC 7231 would seem to quite clearly cover such requests, RFC 7231 was published in 2014. HttpRequestValidationException and the handling of it goes back to at least 2003, when RFC 2616 was the relevant RFC. Under the description of 400 in that RFC, 400 could be read as only relating to requests where the actual HTTP message was mal-formed. While some people did use 400 more generally as a catch-all "client did something wrong" response, it wasn't completely clear that doing so was correct.
Even this though gives a reason why 500 may have been a more appropriate reaction 1.5 years ago, not why it should remain. Changing it though would have backwards-compatibility implications so may not be a good idea.
Still, if a given application that you are writing makes conscious use of HttpRequestValidationException (rather than overriding entirely with your own validation that uses a different approach) then it's sensible to catch such exceptions and send a 400 rather than a 500.
This is just how MVC treat HttpRequestValidationException, I don't like it as well but you can manually change it to a BadRequest:
protected void Application_Error()
{
var lastError = Server.GetLastError();
if (lastError is ArgumentException || lastError is HttpRequestValidationException)
{
Server.ClearError();
Response.StatusCode = (int) HttpStatusCode.BadRequest;
}
}
I used the below code in my c# web service project which is a MVC WebAPI. When client calls my service I have three possibilities . One is there is response to return, second one is there is no response to return because "No configuration found" and third one is also no response because "Same configuration is found".
Now my question is can I use the same HttpStatusCode.NoContent for third scenario also by just changing the message ? So the client who receives the response needs to differentiate by message which I feel is not a good solution. I feel I should pick some other status code , but the meaning may not be appropriate.
response.StatusCode = HttpStatusCode.NoContent;
response.ReasonPhrase = "No Configuration Found.";
Can anyone suggest me what is the best approach in this scenario ?
These status codes are HTTP status codes and and are set by W3 with HTTP protocol standarts. This means you can not override and/or change them. They are fixed through all web.
But you are free to throw any of the erronous codes (mostly are 4xx or 5xx format) amd handle them properly on the client-side. As long as you do it properly user will not notify anything wrong.
Since upgrading to the latest version (3.9.24) of SS our "custom" error handling on the client side (.NET clients) has stopped working as expected. We used to rely on the "ResponseDTO" property of the WebServiceException to handle/respond to exceptions. Now that property is a type of "ServiceModel.ErrorResponse" (which is empty) and not our custom response type. When viewing the raw response in Fiddler the data is as expected, but the SS clients are not parsing/handling it the same as before. Is this design still possible in the new api? If not, what would be the proper approach? We need complete control over the presentation of the data, exceptions, and HTTP status codes.
Thanks