We Keep Coding

Redirect of WebRequest in .NET Core

I wrote the following program in C# using .NET 4.7.1:
var req = (HttpWebRequest) WebRequest.Create(myUrl);
req.AllowAutoRedirect = false;
var rsp = req.GetResponse();
Console.WriteLine(rsp.Headers["Location"]);
The site I am requesting from is returning a 301 response, and the "Location" header contains the URL to redirect to.
If I do the exact same thing using .NET Core 2.1, I will instead get a WebException thrown from the call to GetResponse. How can I avoid this?

Based on this, you need to trap it in try/catch block and inspect the WebException:
If you set AllowAutoRedirect, then you will end up not following the
redirect. That means ending up with the 301 response. HttpWebRequest
(unlike HttpClient) throws exceptions for non-successful (non-200)
status codes. So, getting an exception (most likely a WebException) is
expected. So, if you need to handle that redirect (which is HTTPS ->
HTTP by the way), you need to trap it in try/catch block and inspect
the WebException etc. That is standard use of HttpWebRequest.
That is why we recommend devs use HttpClient which has an easier use
pattern.
Something like this:
WebResponse rsp;
try
{
rsp = req.GetResponse();
}
catch(WebException ex)
{
if(ex.Message.Contains("301"))
rsp = ex.Result;
}

I pretty much figured it out while I was working on it, but I thought I'd post it here in case others run into the same issue.
The response is included as part of the exception that is throw, so I was able to get the same behavior as in .NET 4.7.1 by modifying my code to this:
var req = (HttpWebRequest) WebRequest.Create(myUrl);
req.AllowAutoRedirect = false;
try
{
var rsp = req.GetResponse();
Console.WriteLine(rsp.Headers["Location"]);
}
catch (WebException e)
{
var rsp = (HttpWebResponse) e.Response;
if (rsp.StatusCode == HttpStatusCode.Moved ||
rsp.StatusCode == HttpStatusCode.MovedPermanently ||
rsp.StatusCode == HttpStatusCode.Found)
{
Console.WriteLine(rsp.Headers["Location"]);
}
else throw;
}

http response code is OK when it should not be

I have a restful web service. If someone calls a particular method with the wrong key, I return this:
OutgoingWebResponseContext response = WebOperationContext.Current.OutgoingResponse;
response.StatusCode = System.Net.HttpStatusCode.Unauthorized;
response.StatusDescription = "Invalid key provided.";
If I call it using Firefox with Firebug running, send an invalid key and handle the error in my calling code like:
string serviceResponse = "";
try
{
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
string statusCode = response.StatusCode.ToString();
string statusDescription = response.StatusDescription.ToString();
}
}
catch(Exception ex)
{
serviceResponse = ex.Message;
}
the Response headers show:
HTTP:1.1 200 OK
Having handled the error, I can show a message on the screen. However, as soon as the 'using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)' line is hit, an error is thrown and the response object seems to be discarded.
If I don't put the HttpWebResponse code in a try catch block, a 401 unauthorized message shows in the browser but Firebug now shows:
HTTP:1.2 500 Internal Server Error
... which is weird, as I am sending a 401 back.
What is the point, in a Restful web service, of setting the response StatusCode and StatusDescription as, in a browser calling the web service, as soon as a 401 or 500 StatusCode is encountered, an error is thrown and the Response object discarded.
If a 500 or 401 is returned, how are you supposed to read the StatusDescription?

WebClient: Ignore HTTP 500

I am writing a program which retrieves some data from a server, does some operations on it, and saves the output to a csv file. The problem I have is that the server (which I am not responsible for) ALWAYS returns an HTTP 500 internal server error. I have spoken to the team who look after it, and while they're aware of the bug they've said it's not impacting enough for them to resolve.
Is there a way for me to ignore this response in my code and still get at the data?

If you're using HttpWebRequest/Response, this should get you started:
response = null;
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("<url>");
response = (HttpWebResponse)request.GetResponse();
//no error
}
catch (WebException e)
{
if (e.Status == WebExceptionStatus.ProtocolError)
{
response = (HttpWebResponse)e.Response;
if((int)response.StatusCode == 500)
{
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
var result = sr.ReadToEnd();
}
}
}
}

series of httpWebRequests

I'm currently writing a simple app that performs a series of requests to the web server and I've encountered a strange... feature?
I don't need response stream of the request, but only status code. So, for each piece of my data I call my own "Send" method:
public static int Send(string uri)
{
HttpWebRequest request = null;
HttpWebResponse response = null;
try
{
request = (HttpWebRequest)WebRequest.Create(uri);
response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK) return 0;
}
catch (Exception e)
{
if (request != null) request.Abort();
}
return -1;
}
Works fine? Yes, unless I call this function at least twice. Second call of such a function in a row (with the same uri) will ALWAYS result in timeout.
Now, that's odd: if I add request.Abort(); when I return zero (here, when status code is 200) - everything ALWAYS works fine.
So my question is - why? Is it some kind of framework restriction, or maybe the some kind of anti-DOS protection on the particular server (unfortunately, the server is a black box for me)? Or maybe I just don't understand smth in how it all works?

Try to dispose of the web response, you may leak some resources
public static int Send(string uri)
{
HttpWebRequest request = null;
try
{
request = (HttpWebRequest)WebRequest.Create(uri);
using (var response = (HttpWebResponse)request.GetResponse())
{
if (response.StatusCode == HttpStatusCode.OK) return 0;
}
}
catch (Exception e)
{
if (request != null) request.Abort();
}
return -1;
}
There is also a default number of connections (2 I think, but you can configure this) you can make to a domain simultaneously, please see this SO question. You're probably hitting this limit with your unclosed responses.

First of all I'd make a series of changes in order to get to the root of this:
take out that try..catch{} (you're likely swallowing an exception)
return a boolean instead of a number
You should then get your exception information you need.
Also you should be using "HEAD" as your method as you only want the status code:
request.Method = "HEAD";
read the difference here.

Getting connection closed error when trying to read response from FitNesse REST URI

Have you run into this problem? I run code remarkably similar to that from a this previous question, When in nUnitTest mode and the URI includes "/?test&format=xml" the nUnit test fails with and IOException, "Unable to read data from the transport connection: The connection is closed."
However the Fiddler trace that was running at the time shows the very xml I expected.
I've recreated the request headers exactly (almost) as they are sent when sent through the browser.
Finally, if I leave off the "/?test&format=xml" from the URI, I get the html I would have otherwise expected.
SOURCE CODE:
public virtual bool Run()
{
var request = CreateRequest();
var response = GetResponse(request);
var responseString = ReadResponse(response);
this.SetResults(responseString);
return this.IsSuccessful;
}
protected internal virtual HttpWebRequest CreateRequest()
{
var address = TestConfig.Address;
var request = (HttpWebRequest)WebRequest.Create(address);
request.Accept = "*/*";
request.UseDefaultCredentials = true;
request.CachePolicy = new RequestCachePolicy(RequestCacheLevel.NoCacheNoStore);
return request;
}
protected internal virtual HttpWebResponse GetResponse(HttpWebRequest request)
{
var response = (HttpWebResponse) request.GetResponse();
return response;
}
protected internal virtual string ReadResponse(HttpWebResponse response)
{
var stream = response.GetResponseStream();
var responseString = ReadResponse(stream);
stream.Close();
response.Close();
return responseString;
}
protected internal virtual string ReadResponse(Stream stream)
{
var reader = new StreamReader(stream);
var responseString = reader.ReadToEnd();
return responseString;
}

The error message "Unable to read data from the transport connection: The connection is closed." doesn't really tie up with the fact you're seeing Fiddler getting a html response body back.
Check the StatusCode of the HttpWebResponse (should be 200 if ok), also wrap the request in try/catch block (example from http://msdn.microsoft.com/en-us/library/system.net.httpwebresponse.statuscode(v=vs.80).aspx)
try
{
// Creates an HttpWebRequest for the specified URL.
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(url);
// Sends the HttpWebRequest and waits for a response.
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
if (myHttpWebResponse.StatusCode == HttpStatusCode.OK)
Console.WriteLine("\r\nResponse Status Code is OK and StatusDescription is: {0}",
myHttpWebResponse.StatusDescription);
// Releases the resources of the response.
myHttpWebResponse.Close();
}
catch(WebException e)
{
Console.WriteLine("\r\nWebException Raised. The following error occured : {0}",e.Status);
}
catch(Exception e)
{
Console.WriteLine("\nThe following Exception was raised : {0}",e.Message);
}
If you're creating and disposing of the HttpWebRequest object quickly you might be getting the socket going into a time_wait state as it is shutting down, then you can't re-open it again until it has completely closed. If this is the case, look into using another port or changing the time the connection lives for.