I have an application which sometimes must do some requests to the server to see that those requests are properly blocked. In other words, the expected server answer is 403 Forbidden.
With a piece of code like this:
HttpWebRequest httpRequest = (HttpWebRequest)WebRequest.Create(protectedPage);
httpRequest.Method = WebRequestMethods.Http.Head;
WebResponse response = HttpRequest.GetResponse();
a WebException is thrown on the last line.
When profiling code, this exception in a try/catch block has a performance impact which I want to avoid.
Is there a way to check for server response, expecting a 403, without having to catch an exception (but avoiding using sockets directly)?
Unfortunately, this is a vexing exception in the framework, and there's no way to avoid it (while still using the HttpWebRequest class). But, as High pointed out, this exception handling will take nanoseconds, while the web request itself will take milliseconds (at best), so don't worry about it.
The WebException object has a Response property that contains the response; when you handle the exception, it's important to dispose that WebResponse object (to avoid leaking connections).
What's taking time in the profiling is probably not the exception catching part but the actual response from the web site.
The WebException.Status property will give you the status code for the HTTP response and you can do further processing based on that.
Related
When I tested my web-app based on MVC5 I found that HttpRequestValidationException returns 500 from GetHttpCode().
I catch this exception during security testing of requests to the server. MVC performs request validation and throws this exception, I processed it in the Application_Error and got 500 code returned, but, in fact, if user passed wrong input to server I should return 400 error (bad request). so don't understand why the exception has 500 error. Where I am wrong ?
There are four reasons why 500 might be considered more appropriate.
One is that it isn't clear when a HttpRequestValidationException happens who did the wrong thing. It could be that the user quite clearly sent something incorrect, but it's also possible that the server incorrectly rejected something that was perfectly fine. Considering that this is the default behaviour on many requests that could be malicious, it's probable that the majority of cases where its thrown in response to non-malicious requests (the user isn't actively trying to do the site harm) fall into this category. Hence it could either be something that should be consider 4xx or 5xx and 5xx is a good assumption (don't blame the client when you aren't sure it's the client's fault).
When it happens in a non-malicious case it generally needs attention from a developer, which is only sometimes true of the 4xx codes (a 404 might need developer-attention because the 404 was caused by a bad link on the same site, but it might not). The sort of feedback one gets from 500 responses is more appropriate.
The general model ASP.NET started with pretty much mapped all exceptions to 500 responses, which does make sense a lot of the time. MVC has (sensibly) replaced that with more exceptions that map to other error conditions, but HttpRequestValidationException predates MVC.
Now those points made, I can't say I agree with any of them. I'd still say that HttpRequestValidationException means a client error and if it was mis-identified as such, well it was still identified as such, so it's clearly a 400.
However, the final reason is:
While the description of 400 status in RFC 7231 would seem to quite clearly cover such requests, RFC 7231 was published in 2014. HttpRequestValidationException and the handling of it goes back to at least 2003, when RFC 2616 was the relevant RFC. Under the description of 400 in that RFC, 400 could be read as only relating to requests where the actual HTTP message was mal-formed. While some people did use 400 more generally as a catch-all "client did something wrong" response, it wasn't completely clear that doing so was correct.
Even this though gives a reason why 500 may have been a more appropriate reaction 1.5 years ago, not why it should remain. Changing it though would have backwards-compatibility implications so may not be a good idea.
Still, if a given application that you are writing makes conscious use of HttpRequestValidationException (rather than overriding entirely with your own validation that uses a different approach) then it's sensible to catch such exceptions and send a 400 rather than a 500.
This is just how MVC treat HttpRequestValidationException, I don't like it as well but you can manually change it to a BadRequest:
protected void Application_Error()
{
var lastError = Server.GetLastError();
if (lastError is ArgumentException || lastError is HttpRequestValidationException)
{
Server.ClearError();
Response.StatusCode = (int) HttpStatusCode.BadRequest;
}
}
I have stumbled across an insane behavior of WebClient where it throws WebException on 404 code.
This behavior is wrong because the 404 code can be returned with content data like in the case of Facebook: https://graph.facebook.com/sadfsa.dsadsasadsa
So this code throws an exception
var json = webClient.DownloadString("https://graph.facebook.com/sadfsa.dsadsasadsa");
How do I read the json returned by 404 Facebook?
I don't think that WebClient has insane behavior. Try to catch WebException and read webException.Response.GetResponseStream
See a similar question: C# WebException how to get whole response with a body?
Basically what I've been trying to do is download a file off a server. The server sends a redirect automatically which is fine, but through packet sniffing a program that does successfully download the file I've found that the Headers (for the second request) are:
GET /path/to/file.txt
...
Host: server.com
Rather than the current response being generated (what I thought was standard):
GET www.server.com/path/to/file.txt
Using the normal HttpWebRequest method results in a 500 server error, and I get exceptions thrown when trying to use just the relative path as one would expect.
Using AllowAutoRedirect does not work for this scenario as the cookies are not handled properly, but even if I handle it manually the same error occurs.
How does one go about doing this (preferably without sockets :D)?
To be honest, I'm really not sure what you are asking, but you mentioned cookie troubles. As a total shot in the dark guess, are you setting the CookieContainer on your WebRequest?
request.CookieContainer = new CookieContainer();
request.AllowAutoRedirect = true;
I am writing a program to crawl Web pages using C# HTTTWebRequest. Since I need to monitor the updates of a specific URL, I write the code as follows. However, I noticed that I could get responses for two times only. After that, it got no responses. Even though I lengthen the period to one minute, I still got the same problem. I don't understand why. Could you give me a hand? Thanks so much!
GreatFree
while (true)
{
WebRequest http = HttpWebRequest.Create("http://www.sina.com.cn");
HttpWebResponse response = (HttpWebResponse)http.GetResponse();
Console.WriteLine(response.LastModified);
Thread.Sleep(5000);
}
imaximchuk's answer is correct, but it doesn't explain why.
HTTP connections are pooled - one connection will be reused for a new request if it's available - and returning the connection to the pool occurs when you close or dispose the web response. In your case, you're not closing or disposing the response, so it's not being returned to the pool. There's a maximum number of connections to any particular host, which is configurable but usually shouldn't be changed. The default is two connections - which is why you're seeing two working responses and then a timeout.
I believe there's a finalizer somewhere which will release the connection when it notices the relevant object is eligible for garbage collection, but that's non-deterministic.
Basically: always close a web response, ideal using a using statement.
while (true)
{
WebRequest http = WebRequest.Create("http://www.sina.com.cn");
using (HttpWebResponse response = (HttpWebResponse)http.GetResponse())
{
Console.WriteLine(response.LastModified);
}
Thread.Sleep(5000);
}
You should call response.Close() when you have finishied working with respose or use using(HttpWebResponse response ... to close it automatically
I am hitting up a server with the following code and am encountering a ServerProtocolViolation error:
// Prepare the webpage
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(url + queryString);
// execute the request
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Does anyone know how to work around this kind of error?
This error means that the webserver that you're sending the request to isn't conforming to the HTTP standard.
Other than fixing the server or rewriting HttpWebRequest to be more generous, there isn't much you can do.
What URL are you requesting, and what's the text of the exception?
EDIT: If you request the URL in Fiddler, you'll see that the server didn't return any headers. You should contact the owner of the server and complain.
As a workaround, if you run Fiddler while sending the request, Fiddler will fix the response and allow HttpWebResponse to parse it.
Just a quick note on another reason that I experienced:
If you configure your request to use a proxy server (i.e. through the HttpWebRequest.Proxy property), and you use a wrong proxy port, there might also be a chance to see that error.
In my case, I configured http://127.0.0.1/ as the proxy but had the actual proxy server running on http://127.0.0.1:808/ instead (i.e. port "808" instead of "80").
If this is the case for you, try using no proxy or of course, configure the correct proxy port.