We Keep Coding

How optimize this HTTP connection?

I'm pretty newbie of C# programming and I'm trying to understand how to optimize this piece of code that is called always into a thread to grab image (snapshot) form an IP camera. I'm trying to make some optimization to make faster the HTTP request but I can't see how and where act. May I ask to experts if there is a way to get this result? Thank you.
private Image GetImage()
{
HttpWebRequest req = null;
Image img = null;
HttpWebResponse res = null;
try
{
req = WebRequest.Create(_httpCommand) as HttpWebRequest;
req.Method = "GET";
req.PreAuthenticate = true;
req.AllowWriteStreamBuffering = true;
req.Credentials = new NetworkCredential(_user, _pwd);
res = req.GetResponse() as HttpWebResponse;
using (Stream rs = res.GetResponseStream())
{
img = new Bitmap(rs);
}
}
catch (Exception e) {
_status = false; //<<------------------Signaling the FAILURE status! ---
if (img != null)
img.Dispose(); //<< for ------GC--------------
//insert in LOG
Log logItem = new Log(DateTime.Now, e.Message);
AddLogItem(logItem);
}
return img;
}

I remember, that HttpWebRequest will try to determine proxy settings automatically, which takes time.
req.Proxy = GlobalProxySelection.GetEmptyWebProxy();
tells it to not use any proxy, thus searching for the correct setting will be skipped.
http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.proxy%28v=vs.110%29.aspx

WebRequest and protocol agnostic URL

In a web application I have to first check if an image exists and then display this image or a dummy image.
I use the following code and it works for URLS like:
"http://www.somedomain.com/niceimage.png"
"https://www.somedomain.com/niceimage.png".
public virtual bool WebResourceExists(string url)
{
WebHeaderCollection headers = null;
WebResponse response = null;
try
{
WebRequest request = WebRequest.Create(url);
request.Method = "HEAD";
response = request.GetResponse();
headers = response.Headers;
bool result = int.Parse(headers["Content-Length"]) > 0;
return result;
}
catch (System.Net.WebException)
{
return false;
}
catch (Exception e)
{
_log.Error(e);
return false;
}
finally
{
if (response != null)
{
response.Close();
}
}
}
In some places the the method is called with protocol agnostic urls like "//www.somedomain.com/niceimage.png".
There is an exception thrown for such urls:
System.InvalidCastException: Unable to cast object of type
'System.Net.FileWebRequest' to type 'System.Net.HttpWebRequest'
Is there a way to use protocol agnostic urls other then just prepending "http:" to the url?

Protocol-agnostic URLs are resolved by the browser using the current protocol, and are used to avoid making HTTP requests from an HTTPS page.
Code executing on the server doesn't really have a concept of a "current protocol". Whilst ASP.NET can determine whether the current request was issued over HTTP or HTTPS, the WebRequest classes are not restricted to ASP.NET applications, so they cannot rely on this.
You will need to specify the protocol. Whether you use HTTP or HTTPS will depend on whether you're concerned about third-parties eavesdropping on the connection between your server and "www.somedomain.com".

What about a two step process, check for the http version, if it doesn't exist check for https. I've quickly hacked together a basic example of how this could work but am not able to properly test and check it so it might need some tidying up/refactoring!
public virtual bool WebResourceExists(string url)
{
WebHeaderCollection headers = null;
WebResponse response = null;
try
{
if (url.StartsWith(#"//") {
url = "http:";
}
WebRequest request = WebRequest.Create(url);
request.Method = "HEAD";
response = request.GetResponse();
headers = response.Headers;
bool result = int.Parse(headers["Content-Length"]) > 0;
return result;
}
catch (System.Net.WebException)
{
if (url.StartsWith(#"http://") {
url = url.Replace("http://","https://");
} else {
return false;
}
try {
WebRequest request = WebRequest.Create(url);
request.Method = "HEAD";
response = request.GetResponse();
headers = response.Headers;
bool result = int.Parse(headers["Content-Length"]) > 0;
return result;
}
catch (System.Net.WebException)
{
return false;
}
}
catch (Exception e)
{
_log.Error(e);
return false;
}
finally
{
if (response != null)
{
response.Close();
}
}
}

Keeping a session when using HttpWebRequest

In my project I'm using C# app client and tomcat6 web application server.
I wrote this snippet in the C# client:
public bool isServerOnline()
{
Boolean ret = false;
try
{
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(VPMacro.MacroUploader.SERVER_URL);
req.Method = "HEAD";
req.KeepAlive = false;
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
if (resp.StatusCode == HttpStatusCode.OK)
{
// HTTP = 200 - Internet connection available, server online
ret = true;
}
resp.Close();
return ret;
}
catch (WebException we)
{
// Exception - connection not available
Log.e("InternetUtils - isServerOnline - " + we.Status);
return false;
}
}
Everytime I invoke this method, I get a new session at server side.
I suppose it's because I should use HTTP cookies in my client. But I don't know how to do that, can you help me?

You must use a CookieContainer and keep the instance between calls.
private CookieContainer cookieContainer = new CookieContainer();
public bool isServerOnline()
{
Boolean ret = false;
try
{
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(VPMacro.MacroUploader.SERVER_URL);
req.CookieContainer = cookieContainer; // <= HERE
req.Method = "HEAD";
req.KeepAlive = false;
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
if (resp.StatusCode == HttpStatusCode.OK)
{
// HTTP = 200 - Internet connection available, server online
ret = true;
}
resp.Close();
return ret;
}
catch (WebException we)
{
// Exception - connection not available
Log.e("InternetUtils - isServerOnline - " + we.Status);
return false;
}
}

Check to see if site is online, limit the timeout

I'm trying to make a function that checks if a site is online or not, but is having some problem with the timeout. I want to limit it to a max 3 sec, if there is no respons within 3 sec I should see the page as offline.
My try:
class OnlineCheck
{
public static bool IsOnline(string url)
{
try
{
WebClient webclient = new WebClient();
webclient.Headers.Add(HttpRequestHeader.KeepAlive, "1000");
webclient.OpenRead(url);
}
catch { return false; }
return true;
}
}

The WebClient doesn't support timeout. But you can use the HttpWebRequest!
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(Endpoint);
request.Timeout=3000;
request.GetResponse();

If you want to check that the site is online, you are not really interested in the content of the page, just that you get a response. To make that more efficient, you should only request the http headers. Here is a quick example on how you could do:
private static IEnumerable<HttpStatusCode> onlineStatusCodes = new[]
{
HttpStatusCode.Accepted,
HttpStatusCode.Found,
HttpStatusCode.OK,
// add more codes as needed
};
private static bool IsSiteOnline(string url, int timeout)
{
HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
{
if (request != null)
{
request.Method = "HEAD"; // get headers only
request.Timeout = timeout;
using (var response = request.GetResponse() as HttpWebResponse)
{
return response != null && onlineStatusCodes.Contains(response.StatusCode);
}
}
}
return false;
}

Use HttpWebRequest rather than WebClient. HttpWebRequest class has a timeout property.

You can try this code:
System.Net.WebRequest r = System.Net.WebRequest.Create("http://www.google.com");
r.Timeout = 3000;
System.Net.WebProxy proxy = new System.Net.WebProxy("<proxy address>");
System.Net.NetworkCredential credentials = new System.Net.NetworkCredential();
credentials.Domain = "<domain>";
credentials.UserName = "<login>";
credentials.Password = "<pass>";
proxy.Credentials = credentials;
r.Proxy = proxy;
try
{
System.Net.WebResponse rsp = r.GetResponse();
}
catch (Exception)
{
MessageBox.Show("Is not avaliable");
return;
}
MessageBox.Show("Avaliable!");

static bool isOnline (string URL)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(URL);
request.Timeout = 3000;
try
{
WebResponse resp = request.GetResponse();
}
catch (WebException e)
{
if (((HttpWebResponse)e.Response).StatusCode == HttpStatusCode.NotFound)
{
return false;
}
}
return true;
}

can I check if a file exists at a URL?

I know I can locally, on my filesystem, check if a file exists:
if(File.Exists(path))
Can I check at a particular remote URL?

If you're attempting to verify the existence of a web resource, I would recommend using the HttpWebRequest class. This will allow you to send a HEAD request to the URL in question. Only the response headers will be returned, even if the resource exists.
var url = "http://www.domain.com/image.png";
HttpWebResponse response = null;
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "HEAD";
try
{
response = (HttpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
/* A WebException will be thrown if the status of the response is not `200 OK` */
}
finally
{
// Don't forget to close your response.
if (response != null)
{
response.Close();
}
}
Of course, if you want to download the resource if it exists it would most likely be more efficient to send a GET request instead (by not setting the Method property to "HEAD", or by using the WebClient class).

If you want to just copy & paste Justin's code and get a method to use, here's how I've implemented it:
using System.Net;
public class MyClass {
static public bool URLExists (string url) {
bool result = false;
WebRequest webRequest = WebRequest.Create(url);
webRequest.Timeout = 1200; // miliseconds
webRequest.Method = "HEAD";
HttpWebResponse response = null;
try {
response = (HttpWebResponse)webRequest.GetResponse();
result = true;
} catch (WebException webException) {
Debug.Log(url +" doesn't exist: "+ webException.Message);
} finally {
if (response != null) {
response.Close();
}
}
return result;
}
}
I'll keep his observation:
If you want to download the resource, and it exists, it would be more efficient to send a GET request instead by not setting the Method property to "HEAD" or by using the WebClient class.

Below is a simplified version of the code:
public bool URLExists(string url)
{
bool result = true;
WebRequest webRequest = WebRequest.Create(url);
webRequest.Timeout = 1200; // miliseconds
webRequest.Method = "HEAD";
try
{
webRequest.GetResponse();
}
catch
{
result = false;
}
return result;
}

If you are using a unc path or a mapped drive, this will work fine.
If you are using a web address (http, ftp etc) you are better off using WebClient - you will get a WebException if it doesn't exist.

public static bool UrlExists(string file)
{
bool exists = false;
HttpWebResponse response = null;
var request = (HttpWebRequest)WebRequest.Create(file);
request.Method = "HEAD";
request.Timeout = 5000; // milliseconds
request.AllowAutoRedirect = false;
try
{
response = (HttpWebResponse)request.GetResponse();
exists = response.StatusCode == HttpStatusCode.OK;
}
catch
{
exists = false;
}
finally
{
// close your response.
if (response != null)
response.Close();
}
return exists;
}

I had the same problem to solve in asp.net core, I've solved with HttpClient
private async Task<bool> isFileExist(string url)
{
using (HttpClient client = new HttpClient())
{
var restponse = await client.GetAsync(url);
return restponse.StatusCode == System.Net.HttpStatusCode.OK;
}
}

My version:
public bool IsUrlExist(string url, int timeOutMs = 1000)
{
WebRequest webRequest = WebRequest.Create(url);
webRequest.Method = "HEAD";
webRequest.Timeout = timeOutMs;
try
{
var response = webRequest.GetResponse();
/* response is `200 OK` */
response.Close();
}
catch
{
/* Any other response */
return false;
}
return true;
}

WebRequest will waiting long time(ignore the timeout user set) because not set proxy, so I change to use RestSharp to do this.
var client = new RestClient(url);
var request = new RestRequest(Method.HEAD);
request.Timeout = 5000;
var response = client.Execute(request);
result = response.StatusCode == HttpStatusCode.OK;

Thanks for all answers.
And I would like to add my implementation which includes default state when we get errors, for specific cases like mine.
private bool HTTP_URLExists(String vstrURL, bool vResErrorDefault = false, int vTimeOut = 1200)
{
bool vResult = false;
WebRequest webRequest = WebRequest.Create(vstrURL);
webRequest.Timeout = vTimeOut; // miliseconds
webRequest.Method = "HEAD";
HttpWebResponse response = null;
try
{
response = (HttpWebResponse)webRequest.GetResponse();
if (response.StatusCode == HttpStatusCode.OK) vResult = true;
else if (response.StatusCode == HttpStatusCode.NotFound) vResult = false;
else vResult = vResErrorDefault;
}
catch (WebException ex)
{
if (ex.Status == WebExceptionStatus.ProtocolError && ex.Response != null)
{
var resp01 = (HttpWebResponse)ex.Response;
if (resp01.StatusCode == HttpStatusCode.NotFound)
{
vResult = false;
}
else
{
vResult = vResErrorDefault;
}
}
else
{
vResult = vResErrorDefault;
}
}
finally
{
// Don't forget to close your response.
if (response != null)
{
response.Close();
}
}
return vResult;
}

Anoter version with define timeout :
public bool URLExists(string url,int timeout = 5000)
{
...
webRequest.Timeout = timeout; // miliseconds
...
}

This works for me:
bool HaveFile(string url)
{
try
{
using (WebClient webClient = new WebClient())
{
webClient.DownloadString(url);
}
return true;
}
catch (Exception)
{
return false;
}
}