We Keep Coding

C# webrequests->getresponse casts exception instead of returning status code

I'm attempting to make an image scraper, now for some pages the image was not specified so i wanted to parse my output based on the status code i receive when accessing the page but when i attempt to pare my status code i get an exception instead of the status code if the page was not found, any idea why this happends?
if (gameinfo != null)
if (!string.IsNullOrEmpty(gameinfo.image_uri))
try
{
using (System.Net.WebClient client = new System.Net.WebClient())
{
// Build Uri and attempt to fetch a response.
UriBuilder uribuild = new UriBuilder(gameinfo.image_uri);
WebRequest request = WebRequest.Create(uribuild.Uri);
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
switch (response.StatusCode)
{
// Page found and valid entry.
case HttpStatusCode.OK:
using (Stream stream = client.OpenRead(uribuild.Uri))
{
Console.WriteLine(String.Format("Downloading {0}", uribuild.Uri));
System.Drawing.Bitmap bitmap = new System.Drawing.Bitmap(stream);
bitmap.Save(System.IO.Path.Combine(rom_root, String.Format("{0}.jpg", file_name.Substring(2).Split('.').First())));
}
break;
// Unspecified status codes.
default:
Console.WriteLine("Unspecified status code found, aborting...");
break;
}
}
} catch(System.Net.WebException ex)
{
// Should be moved to switch with HttpStatusCode.NotFound ^
Console.WriteLine("Image page not found.");
}

That's just how the GetResponse() implementation is. If the response is anything other than a success, a WebException is thrown.
I agree, I find it a bit odd - it would be nice if it was optional behaviour at least.
Thankfully, you can read the status code from the WebException which is being thrown:
....
catch (WebException e)
{
using (WebResponse response = e.Response)
{
HttpWebResponse httpResponse = (HttpWebResponse) response;
var statusCode = httpResponse.StatusCode;
// Do stuff with the statusCode
}
}

C# GetResponse not coming back

I have an strange problem. My .NET DataController send a response to an Java server (REST). All works fine, but when I send a response very often, suddenly the GetResponse method not coming back just hangs.
I don't know what's the problem is.
Here is my code
ServicePointManager.DefaultConnectionLimit = 20;
public string HttpGet(string url, string requestAccept)
{
if(string.IsNullOrEmpty(url))
throw new Exception("HttpGet: no REST service URL provided");
if(string.IsNullOrEmpty(requestAccept))
requestAccept = defaultRequestAccept;
if(!url.Equals("pc/alive"))
{
if(string.IsNullOrEmpty(SessionId))
{
if(string.IsNullOrEmpty(Authorization()))
throw new WebException("HttpGet: no login");
}
}
int tries = RestAccessData.MaxReconnect;
string result = string.Empty;
do
{
try
{
var request = NewWebRequest(url, "GET", false, true, requestAccept);
using(var response = request.GetResponse() as HttpWebResponse)
{
UpdateSessionId(response);
HttpStatusCode statusCode = response.StatusCode;
LOG.Debug("...StatusCode: {0} ({1})", (int) response.StatusCode, response.StatusDescription);
if((int) response.StatusCode < 500)
lastContactWithServer = DateTime.Now.Ticks;
switch(statusCode)
{
// Informational 1xx
case HttpStatusCode.Continue: // 100
case HttpStatusCode.SwitchingProtocols: // 101
throw new HttpResponseException(response);
// Successful 2xx
case HttpStatusCode.OK: // 200
result = ReadContent(response);
response.Close();
return (result);
case HttpStatusCode.NoContent: // 204 The server has fulfilled the request but does not need to return an entity-body, and might want to return updated metainformation.
response.Close();
return (string.Empty);
case HttpStatusCode.Created: // 201
case HttpStatusCode.NonAuthoritativeInformation: // 203
result = ReadContent(response);
throw new HttpResponseException(response, result);
case HttpStatusCode.Accepted: // 202 The request has been accepted for processing, but the processing has not been completed.
case HttpStatusCode.ResetContent: // 205 The server has fulfilled the request and the user agent SHOULD reset the document view which caused the request to be sent.
case HttpStatusCode.PartialContent: // 206 The server has fulfilled the partial GET request for the resource.
throw new HttpResponseException(response);
case HttpStatusCode.Unauthorized:
throw new HttpResponseException(response);
default:
throw new HttpResponseException(response);
}
}
}
catch(WebException ex)
{
HandleWebException(ex);
}
catch(HttpResponseException ex)
{
throw ex;
}
catch(SystemException ex)
{
LOG.Error(ex, "caused a(n) {0}", ex.GetType().Name);
}
catch(Exception ex)
{
LOG.Warn(ex, "HttpGet: An error occured while trying to contact the server. Reason: {0}", ex.Message);
throw new UserException("An error occured while trying to contact the server.", ex);
}
}
while(0 < tries--);
return (string.Empty);
}
private HttpWebRequest NewWebRequest(string urlAsString, string requestType, bool withContentType, bool applicationJson, string requestAccept)
{
urlAsString = string.Format("{0}{1}", RestAccessData.GetUrl(), EncodeUrl(urlAsString));
Uri url;
if(!Uri.TryCreate(urlAsString, UriKind.Absolute, out url))
throw new NotSupportedException("url is not compatible");
LOG.Info("RESTUrl {0}", urlAsString);
try
{
var request = HttpWebRequest.Create(url) as HttpWebRequest;
if(!string.IsNullOrEmpty(SessionId))
{
CookieContainer cookies = new CookieContainer();
Cookie cookie = new Cookie
{
Name = "JSESSIONID",
Value = SessionId,
Domain = url.Host
};
cookies.Add(cookie);
request.CookieContainer = cookies;
}
request.Timeout = RestAccessData.Timeout;
request.Method = requestType;
request.AllowAutoRedirect = true;
request.AllowWriteStreamBuffering = true;
request.KeepAlive = false;
request.ContentLength = 0;
request.Headers["Accept-Charset"] = "utf-8";
request.Accept = requestAccept;
request.UserAgent = LvsClient;
if(withContentType)
request.ContentType = applicationJson ? defaultApplicationJson : defaultApplicationWwwForm;
if(!string.IsNullOrEmpty(credential.UserName) && !string.IsNullOrEmpty(credential.Password))
request.Headers["Authorization"] = string.Format("Basic {0}", Convert.ToBase64String(Encoding.UTF8.GetBytes(string.Format("{0}:{1}", credential.UserName, credential.Password))));
return (request);
}
catch(Exception ex)
{
throw new UserException(TH.Translate("NewWebRequest caused an error"), ex);
}
}
Any ideas where the problem is?

This line is making me uncomfortable:
using(var response = request.GetResponse() as HttpWebResponse)
If the response is not an HttpWebResponse then it will never be disposed - the as operator will give you `null.

I found a solution. Very useful is this MSDN article about Understanding MaxServicePointIdleTime and DefaultConnectionLimit. After I play with MaxServicePointIdleTime property and read this comment
change the connection limit at any time
I found a good balance between connection alive and connection close. My client works fine after playing with MaxServicePointIdleTime and DefaultConnectionLimit.

HttpWebResponse get stuck while running in a loop

I build this method (c#) in order to receive the HTTP response status code from an URL.
whene I run this method ones it's works fine, but when I run it in a loop, the third time its stuck. any clue??
public static string isAlive(string url)
{
Console.WriteLine("start: Is Alive Test");
WebRequest request = WebRequest.Create(url);
try
{
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
return Convert.ToString((int)response.StatusCode);
}
catch(WebException ex)
{
HttpWebResponse res = (HttpWebResponse)ex.Response;
return Convert.ToString((int)res.StatusCode);
}
}
the loop
for (int i = 0; i < 5; i++)
{
string a = isAlive("https://www.yahoo.com/");
Console.WriteLine(a);
}

You're not calling Dispose on the HttpWebResponse object, which means that the connection is still lying around. If you change your code to the following:
public static string isAlive(string url)
{
Console.WriteLine("start: Is Alive Test");
WebRequest request = WebRequest.Create(url);
try
{
using(HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
return Convert.ToString((int)response.StatusCode);
}
}
catch(WebException ex)
{
using(HttpWebResponse res = (HttpWebResponse)ex.Response)
{
return Convert.ToString((int)res.StatusCode);
}
}
}
the using statement will implicitly call Dispose for you, which will close the connection.
The reason your code is halting after the second iteration is because .Net has a built in maximum number of connections it will open to a website, which is by default 2. This is controlled by System.Net.ServicePointManager.DefaultConnectionLimit which you can increase should you need to.

You need to wrap HttpWebResponse var into using statement because it's disposable
Before checking ex.Response.StatusCode make sure that ex.Status is a ProtocolError
And also consider making your method asynchronous for performance considerations
Since your method is returning a status code, there might be a better name for it than isAlive
Sample:
public static async Task<string> GetStatusCode(string url)
{
var request = (HttpWebRequest)WebRequest.Create(url);
try
{
using (var response = (HttpWebResponse)await request.GetResponseAsync())
{
return response.StatusCode.ToString();
}
}
catch (WebException ex)
{
return ex.Status == WebExceptionStatus.ProtocolError ?
((HttpWebResponse)e.Response).StatusCode.ToString() : null;
}
}

It might have to do with you not closing the HttpWebResponse.
Add a finally to that try catch which closes the response.
Also close the WebException response within the catch.

Use "using" and it will work well.
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
return Convert.ToString((int)response.StatusCode);
}

Get proper response from http call, not exception

Here is code I use to POST into RESTful web service. My problem is with a last line.
For example, my server can reply with different messages and same code.
Now, unless I get 200 OK I just get exception on last line.
I'd like to have better access to response header, etc no matter what code I got. How is that possible?
var request = WebRequest.Create(Options.DitatServerUri + Options.DitatAccountId + "/integration/trip") as HttpWebRequest;
if (request == null) return false;
request.ContentType = "application/json";
request.Headers.Add(HttpRequestHeader.Authorization, "Basic " + Convert.ToBase64String(Encoding.UTF8.GetBytes(Options.DitatLoginName + ":" + Options.DitatPassword)));
request.Method = "POST";
var serializer = new JavaScriptSerializer();
var serializedData = serializer.Serialize(trip);
var bytes = Encoding.UTF8.GetBytes(serializedData);
request.ContentLength = bytes.Length;
var os = request.GetRequestStream();
os.Write(bytes, 0, bytes.Length);
os.Close();
var response = request.GetResponse();
Example: I get WebException "Invalid Operation" but server actually send message with error explanation.

Building on what Jon said above in the comments, the exception thrown for a bad status code is most likely a WebException, which has Response and Status properties, as per this MSDN page. Therefore, you can get the response via:
WebResponse response = null;
try
{
response = request.GetResponse();
}
catch (WebException ex)
{
response = ex.Response;
}

Why not catch the exception and handle it appropriately?
try
{
var response = request.GetResponse();
}
catch (WebException webEx)
{
Console.WriteLine("Error: {0}", ((HttpWebResponse)webEx.Response).StatusDescription);
}

c# httpwebrequest getResponse() freezes and hangs my program

I was trying to use httpwebrequest to use a rest like service on a remote server and from the first execution itself, my code was hanging the program. Then I tried it as a console application to make sure it has nothing to do with the program itself but no luck!
string credentialsJson = #"{""username"":""test"",
""password"":""test""
}";
int tmp = ServicePointManager.DefaultConnectionLimit;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(#"https://qrua.com/qr/service" + #"/auth/login");
request.Method = "POST";
request.KeepAlive = true;
request.Timeout = 50000 ;
request.CookieContainer = new CookieContainer();
request.ContentType = "application/json";
try
{
StreamWriter writer = new StreamWriter(request.GetRequestStream());
writer.Write(credentialsJson);
}
catch (Exception e)
{
Console.WriteLine("EXCEPTION:" + e.Message);
}
//WebResponse response = request.GetResponse();
try
{
using (WebResponse response = (HttpWebResponse)request.GetResponse())
{
Console.WriteLine("request:\n" + request.ToString() + "\nresponse:\n" + response.ContentLength);
response.Close();
}
}
catch (Exception e)
{
Console.WriteLine("EXCEPTION: in sending http request:" + " \nError message:" + e.Message);
}
Tried several things from different forums but it doesnt help. Even a simple console app with the above code hangs the console indefinitely! Any help would be great..
Thanks

You're never closing the StreamWriter... so I suspect it's not being flushed. Admittedly I'd expect an error from the server instead of just a hang, but it's worth looking at.
Btw, you don't need to close the response and dispose it. Just the using statement is enough.

There's not much you can do if the remote server is not responding other than defining a Timeout and catch the exception as you did in order to inform the user that the operation cannot complete because the remote site didn't respond:
var request = (HttpWebRequest)WebRequest.Create("https://qrua.com/qr/service/auth/login");
request.Timeout = 5000;
// If the server doesn't respond within 5 seconds you might catch the timeout exception
using (var response = request.GetResponse())
{
}
If you don't want to freeze the UI you could use the async version: BeginGetResponse

Try specifying request.ContentLength.
Before doing:
StreamWriter writer = new StreamWriter(request.GetRequestStream());
writer.Write(credentialsJson);
Try something like this:
using (MemoryStream stream = new MemoryStream())
{
using (var writer = StreamWriter writer = new StreamWriter(stream))
{
writer.Write(credentialsJson);
writer.Close();
}
request.ContentLength = stream.Length;
}