When I tested my web-app based on MVC5 I found that HttpRequestValidationException returns 500 from GetHttpCode().
I catch this exception during security testing of requests to the server. MVC performs request validation and throws this exception, I processed it in the Application_Error and got 500 code returned, but, in fact, if user passed wrong input to server I should return 400 error (bad request). so don't understand why the exception has 500 error. Where I am wrong ?
There are four reasons why 500 might be considered more appropriate.
One is that it isn't clear when a HttpRequestValidationException happens who did the wrong thing. It could be that the user quite clearly sent something incorrect, but it's also possible that the server incorrectly rejected something that was perfectly fine. Considering that this is the default behaviour on many requests that could be malicious, it's probable that the majority of cases where its thrown in response to non-malicious requests (the user isn't actively trying to do the site harm) fall into this category. Hence it could either be something that should be consider 4xx or 5xx and 5xx is a good assumption (don't blame the client when you aren't sure it's the client's fault).
When it happens in a non-malicious case it generally needs attention from a developer, which is only sometimes true of the 4xx codes (a 404 might need developer-attention because the 404 was caused by a bad link on the same site, but it might not). The sort of feedback one gets from 500 responses is more appropriate.
The general model ASP.NET started with pretty much mapped all exceptions to 500 responses, which does make sense a lot of the time. MVC has (sensibly) replaced that with more exceptions that map to other error conditions, but HttpRequestValidationException predates MVC.
Now those points made, I can't say I agree with any of them. I'd still say that HttpRequestValidationException means a client error and if it was mis-identified as such, well it was still identified as such, so it's clearly a 400.
However, the final reason is:
While the description of 400 status in RFC 7231 would seem to quite clearly cover such requests, RFC 7231 was published in 2014. HttpRequestValidationException and the handling of it goes back to at least 2003, when RFC 2616 was the relevant RFC. Under the description of 400 in that RFC, 400 could be read as only relating to requests where the actual HTTP message was mal-formed. While some people did use 400 more generally as a catch-all "client did something wrong" response, it wasn't completely clear that doing so was correct.
Even this though gives a reason why 500 may have been a more appropriate reaction 1.5 years ago, not why it should remain. Changing it though would have backwards-compatibility implications so may not be a good idea.
Still, if a given application that you are writing makes conscious use of HttpRequestValidationException (rather than overriding entirely with your own validation that uses a different approach) then it's sensible to catch such exceptions and send a 400 rather than a 500.
This is just how MVC treat HttpRequestValidationException, I don't like it as well but you can manually change it to a BadRequest:
protected void Application_Error()
{
var lastError = Server.GetLastError();
if (lastError is ArgumentException || lastError is HttpRequestValidationException)
{
Server.ClearError();
Response.StatusCode = (int) HttpStatusCode.BadRequest;
}
}
Related
I'm working with a web API that will return code 404 if querying a data that doesn't exist, or other errors if the data is malformed of there's some other problem. Which then results in an HttpRequestException.
Now I'm thinking about a detail. I'm using Polly on that HttpClient connection to ensure it retries in case of communication problems.
In this case, will it work as expected, or will Polly keep retrying in the case of server-thrown errors like "not found" or "bad request"?
I'm configuring it like this
services.AddHttpClient<OntraportHttpClient>()
.AddTransientHttpErrorPolicy(p =>
p.WaitAndRetryAsync(3, _ => TimeSpan.FromMilliseconds(600)));
You have a bit of misunderstanding, 400 Bad Request or 404 Not Found will not result in HttpRequestException.
Unless you call EnsureSuccessStatusCode explicitly.
AddTransientHttpErrorPolicy will check the followings:
408 Timeout
5xx Server error
HttpRequestException
So as you can see neither 400, 404, nor 429 Too Many Requests (typical response code in case of back-pressure) will cause your Polly policy to be triggered. Unless you explicitly call EnsureSuccessStatusCode method.
UPDATE: Adding DELETE use case
Use Case
Let's suppose we have a REST service which exposes a removal functionality of a given resource (addressed by a particular URL and via the DELETE HTTP verb).
This removal can end up in one of the 3 different states from consumption point of view:
Succeeded
Already Done
Failed
You can find several arguments on the internet which is the correct state for succeeded. It can either 200 (OK) with body or 204 (No Content) without body or 202 (Accepted) if it is asynchronous. Sometimes 404 (Not Found) is also used.
The already done state can occur when you try to delete an already deleted item. Without soft deletion it is hard to tell that the given resource has ever existed before or it was never been part of your system. If you have soft deletion, then the service could return 404 for an already deleted resource and 400 (Bad Request) for an unknown resource.
Whenever something fails during the request processing then it can be treated as temporary or permanent failure. If there is a network issue then it can be considered as a temporary/transient issue (this can be manifested as HttpRequestException). If there is a database outage and the service is able to detect it then it can fail fast and return with a 5XX response or it can try to fail over. If there are too many pending requests then the service may consider to throttle them and use back-pressure to shed the load. It might return with 429 (Too Many Requests) along with the appropriate Retry-After header.
Permanent errors, like service has been shut down forever or active refusal of network connection attempts under TLS 1.3 need human intervention to fix them.
Idempotency
Whenever we are talking about retry pattern we need to consider the followings:
The potentially introduced observable impact is acceptable
The operation can be redone without any irreversible side effect
The introduced complexity is negligible compared to the promised reliability
The second criteria is usually referred as Idempotency. It says that if you call the method / endpoint multiple times with the same input then it should return the same output without any side effect.
If your service's removal functionality can be considered as idempotent then there is no such state as Already done. If you call it 100 times then it should always return with "yepp, that's gone". So with this is mind it might make sense to return with either 204 or 404 in case of idempotent deletion.
Resilient strategy
Whenever we are talking about strategy it means for me a chain of resilient policies. If a former policy could not "fix" the problem then the latter would try to do that (so there is a policy escalation).
Server-side: You can use Bulk-head policy to have control over the maximum number of concurrent calls but if the threshold has been exceeded then you can start to throttle requests.
Client-side: You can have a timeout for each individual request and you can apply retry policy in case of temporary/transient failure. You can also define a global timeout for all your retry attempts. Or you can apply a circuit breaker to monitor successive failures and back-off for a given period of time if the service is treated as overwhelmed or malfunctioning.
My 2 cents is applying a single resilient policy on the client-side might not be enough to have a robust and resilient system. It might require several policies (on both sides) to establish a communication protocol for problematic periods.
Is there any possibility to throw an exception in the code of a web application which would not cause error 500 in the IIS logs?
500 means internal server error, i.e. your application have failed.
I'm guessing that you are asking for some sort of indication when the client have done something wrong.
For MVC 5.x there is an HttpException which can be used to return other error codes. (500 is returned for all exceptions but HttpException)
However! Exceptions are application failures. Your application does not fail just because the client sent something invalid.
Instead, you should send back a HttpResponse with the correct HTTP code. For MVC applications you can use the HttpStatusCodeResult in the controller.
I always wonder what is the best error response code in folowing situation:
public IActionResult Index(Guid id)
{
var entity = _dbContext.Entities.Find(id);
if (entity == null)
{
return NotFound(); //??
}
return View(entity)
}
404 - Not Found seems most appropriate, however, from debugging point of view, non-existing ID and wrong controller/action names are very different errors.
Anyway, I've decided to return custom error page with more explanatory message, so I can differentiate between 404 and 404.
How to return custom 404 page from Controller's Action and default 404 in other cases?
I would like to avoid returning HttpResponseMessage, since I would need to change return type of the Action.
PS: In comments you may vote for 404 resp other response code you would use in this particular case.
Take a look at this question first.
Theoreticaly you shouldn't use HttpStatus-codes as Application Error Codes.
However, on a public website, 404 has one specific meaning:
The requested resource could not be found but may be available in the
future. Subsequent requests by the client are permissible.
Calling /getresource/347d2f3a-bd0f-4d0b-8c05-2e7f3f8f265e is a resource. If this is not available you should use 404. If it is available, then 200. That's perfectly fine.
Google even says, you should send 404. From the google support:
Returning a code other than 404 or 410 for a non-existent page (or
redirecting users to another page, such as the homepage, instead of
returning a 404) can be problematic. Firstly, it tells search engines
that there’s a real page at that URL. As a result, that URL may be
crawled and its content indexed. Because of the time Googlebot spends
on non-existent pages, your unique URLs may not be discovered as
quickly or visited as frequently and your site’s crawl coverage may be
impacted (also, you probably don’t want your site to rank well for the
search query
For the implementation, I would just throw a custom NotFoundException (If you want to setup more meaningfull information) with all the data you need, and handle it globaly through the ExceptionFilterAttribute. There you can turn it into a 404 response.
Of course you can let the original exception just bubble up to the ExceptionFilterAttribute, but then you have not that many possibilites for making it meaningfull.
You can utilize the CreateResponse extension method and would do something as follows:
return Request.CreateResponse(HttpStatusCode.NotFound, "foobar");
I've used the HttpStatusCode.NoContent for something like this before. It really depends on what your business logic is and how you want to handle these cases. NoContent will result in a successful http call where as NotFound will be an error. NotFound will trigger your default MVC error page route(if you have that setup) and NoContent won't. Is this a response that is possible through normal use/traversing of the app or is it only something that will occur if a malicious user is tampering with urls? All of those pieces are what I take into consideration determining which http status code I want to return. Hope this helps!
As strange as this may sound, I've been getting a Bad Request (error 400) response from IIS whenever I post a form that has an invalid date in a datetime field, as if IIS knows it's supposed to be a datetime value and is validating it somehow.
UPDATE (added more information)
It's an ASP.NET / MVC 4.0 website.
All datetime fields in the model are declared as string (for other
reasons), so I don't think it's model validation.
This behavior only occurs in one of the servers, all other machines (developer and servers) are ok.
UPDATE 2 (added more information)
It's seems to be a proxy related issue. When accessing the site locally, it works. When accessing it from another computer, returns bad request.
UPDATE 3 (add more information)
Ruled out the "proxy hypothesis". I enabled request tracing in IIS, this was logged:
ModuleName: ManagedPipelineHandler
Notification: 128
HttpStatus: 400
HttpReason: Bad Request
HttpSubStatus: 0
ErrorCode: 0
Notification: EXECUTE_REQUEST_HANDLER
ErrorCode: A operação foi concluída com êxito. (0x0)
I have two posts:
btnProcessarAutuacao=Processar&IdAutuacao=5000038&DataLimiteIdentificacaoCondutor=31%2F12%2F2013+00%3A00%3A00&DscMensagemErro=Numero+do+auto+de+Infra%C3%A7%C3%A3o+A151515+para+o+Orgao+autuador+100107+j%C3%A1+foi+digitalizado&Placa=GVQ3641&AIT=A151515&CodInfracao=7471&DscInfracao=EXC.VELOC.ALEM+50%25+MAX++++++++&CodOrgaoAutuador=100107&DscOrgaoAutuador=GOVERNO+DO+DISTRITO+FEDERAL&DataEmissao=09%2F01%2F2014&DataInfracao=31%2F12%2F2013&HoraAutuacao=10%3A10%3A00&CodMunicipio=643&DscMunicipio=643&Local1=p_local145&Local2=p_local245&X-Requested-With=XMLHttpRequest
2)
btnProcessarAutuacao=Processar&IdAutuacao=5000038&DataLimiteIdentificacaoCondutor=31%2F12%2F2013+00%3A00%3A00&DscMensagemErro=Numero+do+auto+de+Infra%C3%A7%C3%A3o+A151515+para+o+Orgao+autuador+100107+j%C3%A1+foi+digitalizado&Placa=GVQ3641&AIT=A151515&CodInfracao=7471&DscInfracao=EXC.VELOC.ALEM+50%25+MAX++++++++&CodOrgaoAutuador=100107&DscOrgaoAutuador=GOVERNO+DO+DISTRITO+FEDERAL&DataEmissao=31%2F02%2F2014&DataInfracao=31%2F12%2F2013&HoraAutuacao=10%3A10%3A00&CodMunicipio=643&DscMunicipio=643&Local1=p_local145&Local2=p_local245&X-Requested-With=XMLHttpRequest
The second post sends and invalid date (02/31/2014) for the "DataEmissao" parameter and IIS responds with error 400 (Bad Request). The first post has a valid date for the same parameter, IIS responds with OK (200).
That's the only difference I could find in these requests.
Any clues to what is happening?
Finally got it figured out.
I'm doing custom server-side validation, and whenever some field has an invalid value, I set the response code to 400 (Bad Request) and return a custom error message.
When testing locally, my custom error message goes through. When testing remotely, my custom message is replace with a default IIS error page.
All I had to do was add this to web.config:
<system.webServer>
<httpErrors errorMode="Detailed"></httpErrors>
</system.webServer>
So that my custom response text goes without being replaced by the IIS default message.
Got the answer from here: Custom 400 with message being overwritten.
This isn't strange at all :)
IIS doesn't know what the data-type of the query string parameter is supposed to be, but the application framework does. For example, if this is an ASP.NET web-page, then the ASP.NET runtime could be generating the 400, or, alternatively, a programmer could have implemented a check on the type and manually set the return code to 400.
Without more details (and some code!) it's impossible to say precisely where the 400 is being generated, but it will be along these lines...
I need to write code to handle the 2 distinct types of failures that can arise after issuing an HTTP request using an HttpClient object that comes with .Net Framework 4.5:
Connectivity failures, when the server can not be reached due to connection problems - e.g. no connection available, timeout expiration, incorrect network settings etc.
Although technically not a failure, the case when the request successfully reaches the server but the server responds with an HTTP error status code (e.g. 404, 400, 409, 500 etc.)
It is very important that one failure type is not treated as the other. Which is the correct and reliable way to do this? The EnsureSuccessStatusCode() method seems to help with at least the second case, but the documentation is disappointingly scarce.
After some manual testing, the good news is that there seems to be a reliable method to separate the 2 types of errors, but the bad news is that it's not consistently the same on all platforms - at least not on Windows Phone 8. On .Net Framework 4.5 Full and on Windows Store App types of project, the following code does the trick:
try
{
var response = await httpClient.PostAsync(uri, content);
if (!response.IsSuccessStatusCode)
{
// handle the second type of error (404, 400, etc.)
}
}
catch (HttpRequestException ex)
{
// handle the first type of error (no connectivity, etc)
}
On Windows Phone 8, unfortunately, both types are handled under the if condition: IsSuccessStatusCode is set to false and no HttpRequestException (or any other exception, for that matter) is raised. For WP8 I'm using the portable class library implementation of HttpClient (http://blogs.msdn.com/b/bclteam/archive/2013/02/18/portable-httpclient-for-net-framework-and-windows-phone.aspx), I asked them if this inconsistency can be fixed, hopefully it can and will soon be.