We Keep Coding

Check the uri is valid

Uri myuri = new Uri("https://buffalousercontent.blob.core.windows.net/371-static/profileimages/full/39398");
This is my uri, when u hit means it receives image based on user some user doesn't contains image at that time it receives error page.
How to check the received one is image or something in C# code?

If you're just looking to validate that the URL contains an image:
bool ValidateImage(string url)
{
HttpWebRequest r = (HttpWebRequest)WebRequest.Create(url);
r.Method = "GET";
try
{
HttpWebResponse resp = (HttpWebResponse)r.GetResponse();
if (resp.ContentType == "image/jpeg")
{
Console.WriteLine("Image retrieved successfully.");
// Process image
return true;
}
else
{
Console.WriteLine("Unable to retrieve image");
}
}
catch
{
Console.WriteLine("Unable to retrieve image.");
}
return false;
}
Obviously, change the ContentType check to whatever makes sense for your application. You could also use HEAD as the request method (as in #Chris' answer) if you're just looking to validate the content type and not download the entire image right then.

As mentioned in the other answers, you can look at the content type of the response, or you could try to create an image from the URI.
try {
var client = new WebClient();
var image = Image.FromStream(client.OpenRead(uri));
}
catch(ArguementException e) {
// this exception will be thrown if the URI doesn't point to a valid image.
}

To check if there is an error, you have to send a request and then receive an error or a good blob.
To send a request, you can use WebClient as described here :
public bool CheckUri(Uri uri){
using(var client = new WebClient()){
try{
client.DownloadFile(uri);
return true;
}catch{//error detected
return false;
}
}
}

Do a httprequest and check if the response contenttype equals "image/". This function should do the trick!
Boolean IsImageUrl(string URL)
{
var req = (HttpWebRequest)HttpWebRequest.Create(URL);
req.Method = "HEAD";
using (var resp = req.GetResponse())
{
return resp.ContentType.ToLower(CultureInfo.InvariantCulture)
.StartsWith("image/");
}
}

Well a Uri object itself doesn't know what will happen when you use it as the address of a GET request.
To check if a string is a valid Uri, you can use
Uri uri;
bool valid = Uri.TryCreate(stringInput, UriKind.Absolute, out uri);
To check what the file type of a request is
using (var client = new HttpClient())
{
var response = client.GetAsync(uri).Result;
var contentType = response.Content.Headers.ContentType;
}

You can use the Azure client API to do this. Add a reference to Microsoft.WindowsAzure.StorageClient:
bool IsBlobJpeg(string blobUri)
{
try
{
Microsoft.WindowsAzure.StorageClient.CloudBlob blob =
new CloudBlob(
blobUri);
blob.FetchAttributes();
return (blob.Properties.ContentType == "image/jpeg");
}
catch (StorageClientException ex)
{
Console.WriteLine(String.Format("{0} Does not exist", blobUri));
return false;
}
}

What is the best way to use .PAC in C# when request is redirected

Background
I have to use the proxy server specified by users in my application.
Right now, I check that the input proxy contains ".pac" or not. If yes, I'll get the real proxy from pac file. (http://www.codeproject.com/Articles/12168/Using-PAC-files-proxy) Otherwise, I just set the input proxy to WebProxy.
public HttpWebRequest CreateRequest(string url, string proxy)
{
var request = WebRequest.Create(url);
if (proxy.ToLower().Contains(".pac"))
{
string realProxy = GetProxyFromPac(url, proxy);
request.Proxy = (string.IsNullOrEmpty(realProxy))
? new WebProxy()
: new WebProxy(realProxy, true);
}
else
{
request.Proxy = new WebProxy(proxy, true);
}
return request as HttpWebRequest;
}
And use the request like this.
var request = CreateRequest(url, proxy);
var response = request.GetResponse() as HttpWebResponse;
Problem
When the server redirects url to another url, I'll get request timeout.
Example:
The input url URL
The input proxy PAC_P
GetProxyFromPac(PAC_P, URL) return P
The redirected url REAL_URL
GetProxyFromPac(PAC_P, REAL_URL) return PP
I found that it's because I set the proxy to P in the request and it will be used for all urls (including redirected urls) but REAL_URL is not accessible via P.
I have to get PP from the REAL_URL and PAC_P and use PP to request to REAL_URL.
The first solution in my head is to get a new proxy every time the request is redirected and manually request to the redirected url.
var request = CreateRequest(url, proxy);
request.AllowAutoRedirect = false;
var response = request.GetResponse() as HttpWebResponse;
while (response.StatusCode == HttpStatusCode.Redirect ||
response.StatusCode == HttpStatusCode.Moved)
{
request = CreateRequest(response.Headers["Location"], proxy);
response = request.GetResponse() as HttpWebResponse;
}
Question
I think there should be an elegant way to handle this. It seems to be an extremely general case. Do you have any idea?

You can implement your own IWebProxy class like this:
public class MyPacScriptProxy : IWebProxy
{
protected string PacScriptUrl;
public MyPacScriptProxy(string url)
{
PacScriptUrl = url;
}
public ICredentials Credentials { get; set; }
public Uri GetProxy(Uri dest)
{
// you can return your GetProxyFromPac(dest, PacScriptUrl); result here
if (dest.Host.EndsWith(".net")) {
return null; // bypass proxy for .net websites
}
return new Uri("http://localhost:8888");
}
public bool IsBypassed(Uri host)
{
return false;
}
}
then use it like this:
var request = WebRequest.Create("http://www.google.com");
request.Proxy = new MyPacScriptProxy("http://localhost/proxy.pac");
Hope this helps :)

WebRequest and protocol agnostic URL

In a web application I have to first check if an image exists and then display this image or a dummy image.
I use the following code and it works for URLS like:
"http://www.somedomain.com/niceimage.png"
"https://www.somedomain.com/niceimage.png".
public virtual bool WebResourceExists(string url)
{
WebHeaderCollection headers = null;
WebResponse response = null;
try
{
WebRequest request = WebRequest.Create(url);
request.Method = "HEAD";
response = request.GetResponse();
headers = response.Headers;
bool result = int.Parse(headers["Content-Length"]) > 0;
return result;
}
catch (System.Net.WebException)
{
return false;
}
catch (Exception e)
{
_log.Error(e);
return false;
}
finally
{
if (response != null)
{
response.Close();
}
}
}
In some places the the method is called with protocol agnostic urls like "//www.somedomain.com/niceimage.png".
There is an exception thrown for such urls:
System.InvalidCastException: Unable to cast object of type
'System.Net.FileWebRequest' to type 'System.Net.HttpWebRequest'
Is there a way to use protocol agnostic urls other then just prepending "http:" to the url?

Protocol-agnostic URLs are resolved by the browser using the current protocol, and are used to avoid making HTTP requests from an HTTPS page.
Code executing on the server doesn't really have a concept of a "current protocol". Whilst ASP.NET can determine whether the current request was issued over HTTP or HTTPS, the WebRequest classes are not restricted to ASP.NET applications, so they cannot rely on this.
You will need to specify the protocol. Whether you use HTTP or HTTPS will depend on whether you're concerned about third-parties eavesdropping on the connection between your server and "www.somedomain.com".

What about a two step process, check for the http version, if it doesn't exist check for https. I've quickly hacked together a basic example of how this could work but am not able to properly test and check it so it might need some tidying up/refactoring!
public virtual bool WebResourceExists(string url)
{
WebHeaderCollection headers = null;
WebResponse response = null;
try
{
if (url.StartsWith(#"//") {
url = "http:";
}
WebRequest request = WebRequest.Create(url);
request.Method = "HEAD";
response = request.GetResponse();
headers = response.Headers;
bool result = int.Parse(headers["Content-Length"]) > 0;
return result;
}
catch (System.Net.WebException)
{
if (url.StartsWith(#"http://") {
url = url.Replace("http://","https://");
} else {
return false;
}
try {
WebRequest request = WebRequest.Create(url);
request.Method = "HEAD";
response = request.GetResponse();
headers = response.Headers;
bool result = int.Parse(headers["Content-Length"]) > 0;
return result;
}
catch (System.Net.WebException)
{
return false;
}
}
catch (Exception e)
{
_log.Error(e);
return false;
}
finally
{
if (response != null)
{
response.Close();
}
}
}

can I check if a file exists at a URL?

I know I can locally, on my filesystem, check if a file exists:
if(File.Exists(path))
Can I check at a particular remote URL?

If you're attempting to verify the existence of a web resource, I would recommend using the HttpWebRequest class. This will allow you to send a HEAD request to the URL in question. Only the response headers will be returned, even if the resource exists.
var url = "http://www.domain.com/image.png";
HttpWebResponse response = null;
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "HEAD";
try
{
response = (HttpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
/* A WebException will be thrown if the status of the response is not `200 OK` */
}
finally
{
// Don't forget to close your response.
if (response != null)
{
response.Close();
}
}
Of course, if you want to download the resource if it exists it would most likely be more efficient to send a GET request instead (by not setting the Method property to "HEAD", or by using the WebClient class).

If you want to just copy & paste Justin's code and get a method to use, here's how I've implemented it:
using System.Net;
public class MyClass {
static public bool URLExists (string url) {
bool result = false;
WebRequest webRequest = WebRequest.Create(url);
webRequest.Timeout = 1200; // miliseconds
webRequest.Method = "HEAD";
HttpWebResponse response = null;
try {
response = (HttpWebResponse)webRequest.GetResponse();
result = true;
} catch (WebException webException) {
Debug.Log(url +" doesn't exist: "+ webException.Message);
} finally {
if (response != null) {
response.Close();
}
}
return result;
}
}
I'll keep his observation:
If you want to download the resource, and it exists, it would be more efficient to send a GET request instead by not setting the Method property to "HEAD" or by using the WebClient class.

Below is a simplified version of the code:
public bool URLExists(string url)
{
bool result = true;
WebRequest webRequest = WebRequest.Create(url);
webRequest.Timeout = 1200; // miliseconds
webRequest.Method = "HEAD";
try
{
webRequest.GetResponse();
}
catch
{
result = false;
}
return result;
}

If you are using a unc path or a mapped drive, this will work fine.
If you are using a web address (http, ftp etc) you are better off using WebClient - you will get a WebException if it doesn't exist.

public static bool UrlExists(string file)
{
bool exists = false;
HttpWebResponse response = null;
var request = (HttpWebRequest)WebRequest.Create(file);
request.Method = "HEAD";
request.Timeout = 5000; // milliseconds
request.AllowAutoRedirect = false;
try
{
response = (HttpWebResponse)request.GetResponse();
exists = response.StatusCode == HttpStatusCode.OK;
}
catch
{
exists = false;
}
finally
{
// close your response.
if (response != null)
response.Close();
}
return exists;
}

I had the same problem to solve in asp.net core, I've solved with HttpClient
private async Task<bool> isFileExist(string url)
{
using (HttpClient client = new HttpClient())
{
var restponse = await client.GetAsync(url);
return restponse.StatusCode == System.Net.HttpStatusCode.OK;
}
}

My version:
public bool IsUrlExist(string url, int timeOutMs = 1000)
{
WebRequest webRequest = WebRequest.Create(url);
webRequest.Method = "HEAD";
webRequest.Timeout = timeOutMs;
try
{
var response = webRequest.GetResponse();
/* response is `200 OK` */
response.Close();
}
catch
{
/* Any other response */
return false;
}
return true;
}

WebRequest will waiting long time(ignore the timeout user set) because not set proxy, so I change to use RestSharp to do this.
var client = new RestClient(url);
var request = new RestRequest(Method.HEAD);
request.Timeout = 5000;
var response = client.Execute(request);
result = response.StatusCode == HttpStatusCode.OK;

Thanks for all answers.
And I would like to add my implementation which includes default state when we get errors, for specific cases like mine.
private bool HTTP_URLExists(String vstrURL, bool vResErrorDefault = false, int vTimeOut = 1200)
{
bool vResult = false;
WebRequest webRequest = WebRequest.Create(vstrURL);
webRequest.Timeout = vTimeOut; // miliseconds
webRequest.Method = "HEAD";
HttpWebResponse response = null;
try
{
response = (HttpWebResponse)webRequest.GetResponse();
if (response.StatusCode == HttpStatusCode.OK) vResult = true;
else if (response.StatusCode == HttpStatusCode.NotFound) vResult = false;
else vResult = vResErrorDefault;
}
catch (WebException ex)
{
if (ex.Status == WebExceptionStatus.ProtocolError && ex.Response != null)
{
var resp01 = (HttpWebResponse)ex.Response;
if (resp01.StatusCode == HttpStatusCode.NotFound)
{
vResult = false;
}
else
{
vResult = vResErrorDefault;
}
}
else
{
vResult = vResErrorDefault;
}
}
finally
{
// Don't forget to close your response.
if (response != null)
{
response.Close();
}
}
return vResult;
}

Anoter version with define timeout :
public bool URLExists(string url,int timeout = 5000)
{
...
webRequest.Timeout = timeout; // miliseconds
...
}

This works for me:
bool HaveFile(string url)
{
try
{
using (WebClient webClient = new WebClient())
{
webClient.DownloadString(url);
}
return true;
}
catch (Exception)
{
return false;
}
}

Getting the Redirected URL from the Original URL

I have a table in my database which contains the URLs of some websites. I have to open those URLs and verify some links on those pages. The problem is that some URLs get redirected to other URLs. My logic is failing for such URLs.
Is there some way through which I can pass my original URL string and get the redirected URL back?
Example: I am trying with this URL:
http://individual.troweprice.com/public/Retail/xStaticFiles/FormsAndLiterature/CollegeSavings/trp529Disclosure.pdf
It gets redirected to this one:
http://individual.troweprice.com/staticFiles/Retail/Shared/PDFs/trp529Disclosure.pdf
I tried to use following code:
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(Uris);
req.Proxy = proxy;
req.Method = "HEAD";
req.AllowAutoRedirect = false;
HttpWebResponse myResp = (HttpWebResponse)req.GetResponse();
if (myResp.StatusCode == HttpStatusCode.Redirect)
{
MessageBox.Show("redirected to:" + myResp.GetResponseHeader("Location"));
}
When I execute the code above it gives me HttpStatusCodeOk. I am surprised why it is not considering it a redirection. If I open the link in Internet Explorer then it will redirect to another URL and open the PDF file.
Can someone help me understand why it is not working properly for the example URL?
By the way, I checked with Hotmail's URL (http://www.hotmail.com) and it correctly returns the redirected URL.

This function will return the final destination of a link — even if there are multiple redirects. It doesn't account for JavaScript-based redirects or META redirects. Notice that the previous solution didn't deal with Absolute & Relative URLs, since the LOCATION header could return something like "/newhome" you need to combine with the URL that served that response to identify the full URL destination.
public static string GetFinalRedirect(string url)
{
if(string.IsNullOrWhiteSpace(url))
return url;
int maxRedirCount = 8; // prevent infinite loops
string newUrl = url;
do
{
HttpWebRequest req = null;
HttpWebResponse resp = null;
try
{
req = (HttpWebRequest) HttpWebRequest.Create(url);
req.Method = "HEAD";
req.AllowAutoRedirect = false;
resp = (HttpWebResponse)req.GetResponse();
switch (resp.StatusCode)
{
case HttpStatusCode.OK:
return newUrl;
case HttpStatusCode.Redirect:
case HttpStatusCode.MovedPermanently:
case HttpStatusCode.RedirectKeepVerb:
case HttpStatusCode.RedirectMethod:
newUrl = resp.Headers["Location"];
if (newUrl == null)
return url;
if (newUrl.IndexOf("://", System.StringComparison.Ordinal) == -1)
{
// Doesn't have a URL Schema, meaning it's a relative or absolute URL
Uri u = new Uri(new Uri(url), newUrl);
newUrl = u.ToString();
}
break;
default:
return newUrl;
}
url = newUrl;
}
catch (WebException)
{
// Return the last known good URL
return newUrl;
}
catch (Exception ex)
{
return null;
}
finally
{
if (resp != null)
resp.Close();
}
} while (maxRedirCount-- > 0);
return newUrl;
}

The URL you mentioned uses a JavaScript redirect, which will only redirect a browser. So there's no easy way to detect the redirect.
For proper (HTTP Status Code and Location:) redirects, you might want to remove
req.AllowAutoRedirect = false;
and get the final URL using
myResp.ResponseUri
as there can be more than one redirect.
UPDATE: More clarification regarding redirects:
There's more than one way to redirect a browser to another URL.
The first way is to use a 3xx HTTP status code, and the Location: header. This is the way the gods intended HTTP redirects to work, and is also known as "the one true way." This method will work on all browsers and crawlers.
And then there are the devil's ways. These include meta refresh, the Refresh: header, and JavaScript. Although these methods work in most browsers, they are definitely not guaranteed to work, and occasionally result in strange behavior (aka. breaking the back button).
Most web crawlers, including the Googlebot, ignore these redirection methods, and so should you. If you absolutely have to detect all redirects, then you would have to parse the HTML for META tags, look for Refresh: headers in the response, and evaluate Javascript. Good luck with the last one.

Use this code to get redirecting URL
public void GrtUrl(string url)
{
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.AllowAutoRedirect = false; // IMPORTANT
webRequest.Timeout = 10000; // timeout 10s
webRequest.Method = "HEAD";
// Get the response ...
HttpWebResponse webResponse;
using (webResponse = (HttpWebResponse)webRequest.GetResponse())
{
// Now look to see if it's a redirect
if ((int)webResponse.StatusCode >= 300 &&
(int)webResponse.StatusCode <= 399)
{
string uriString = webResponse.Headers["Location"];
Console.WriteLine("Redirect to " + uriString ?? "NULL");
webResponse.Close(); // don't forget to close it - or bad things happen
}
}
}

Here's two Async HttpClient versions:
Works in .Net Framework and .Net Core
public static async Task<Uri> GetRedirectedUrlAsync(Uri uri, CancellationToken cancellationToken = default)
{
using var client = new HttpClient(new HttpClientHandler
{
AllowAutoRedirect = false,
}, true);
using var response = await client.GetAsync(uri, cancellationToken);
return new Uri(response.Headers.GetValues("Location").First();
}
Works in .Net Core
public static async Task<Uri> GetRedirectedUrlAsync(Uri uri, CancellationToken cancellationToken = default)
{
using var client = new HttpClient();
using var response = await client.GetAsync(uri, cancellationToken);
return response.RequestMessage.RequestUri;
}
P.S. handler.MaxAutomaticRedirections = 1 can be used if you need to limit the number of attempts.

After reviewing everyone's suggestions I kind of figured this out for at least my case which basically did 3 loops once to https and second one to actual ending location.
This is a recursive function call here:
public static string GrtUrl(string url, int counter)
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls
| SecurityProtocolType.Tls11
| SecurityProtocolType.Tls12
| SecurityProtocolType.Ssl3;
string ReturnURL = url;
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.AllowAutoRedirect = false; // IMPORTANT
webRequest.Timeout = 10000; // timeout 10s
webRequest.Method = "HEAD";
// Get the response ...
HttpWebResponse webResponse;
using (webResponse = (HttpWebResponse)webRequest.GetResponse())
{
// Now look to see if it's a redirect
if ((int)webResponse.StatusCode >= 300 && (int)webResponse.StatusCode <= 399)
{
string uriString = webResponse.Headers["Location"];
ReturnURL = uriString;
if (ReturnURL == url)
{
webResponse.Close(); // don't forget to close it - or bad things happen!
return ReturnURL;
}
else
{
webResponse.Close(); // don't forget to close it - or bad things happen!
if (counter > 50)
return ReturnURL;
else
return GrtUrl(ReturnURL, counter++);
}
}
}
return ReturnURL;
}

You could check the Request.UrlReferrer.AbsoluteUri to see where i came from. If that doesn't work can you pass the old url as a query string parameter?

This code works for me
var request = (HttpWebRequest)HttpWebRequest.Create(url);
request.Method = "POST";
request.AllowAutoRedirect = true;
request.ContentType = "application/x-www-form-urlencoded";
var response = request.GetResponse();
//After sending the request and the request is expected to redirect to some page of your website, The response.ResponseUri.AbsoluteUri contains that url including the query strings
//(www.yourwebsite.com/returnulr?r=""... and so on)
Redirect(response.ResponseUri.AbsoluteUri); //then just do your own redirect.
Hope this helps

I had the same problem and after tryin a lot I couldn't get what i wanted with HttpWebRequest so i used web browser class to navigate to first url and then i could get the redirected url !
WebBrowser browser = new WebBrowser();
browser.Navigating += new System.Windows.Forms.WebBrowserNavigatingEventHandler(this.browser_Navigating);
string urlToNavigate = "your url";
browser.Navigate(new Uri(urlToNavigate));
then on navigating you can get your redirected url. Be careful that the first time browser_Navigating event handler occurs, e.url is the same url you used to start browsing so you can get redirected url on the second call
private void browser_Navigating(object sender, WebBrowserNavigatingEventArgs e)
{
Uri uri = e.Url;
}

This code worked for me with Unicode support:
public static string GetFinalRedirect(string url)
{
try
{
var request = (HttpWebRequest)HttpWebRequest.Create(url);
request.Method = "POST";
request.AllowAutoRedirect = true;
request.ContentType = "application/x-www-form-urlencoded";
var response = request.GetResponse();
return response.ResponseUri.AbsoluteUri.ToString();
}
catch(Exception ax)
{
return "";
}
}

string url = ".......";
var request = (HttpWebRequest)WebRequest.Create(url);
var response = (HttpWebResponse)request.GetResponse();
string redirectUrl = response.ResponseUri.ToString();

A way to deal with javascript redirect is to view the source code of the initial domain's page that would load and then extract a new domain aka the final domain directly from the source code. Since it is a javascript redirect then the new domain aka final domain should be there. Cheers
Code to extract the URL address from page source:
string href = "";
string pageSrc = "get page source using web client download string method and place output here";
Match m = Regex.Match(pageSrc, #"href=\""(.*?)\""", RegexOptions.Singleline);
if (m2.Success){
href = m.Groups[1].Value; /* will result in http://finalurl.com */
}

I made this method using your code and it returns the final redirected URL.
public string GetFinalRedirectedUrl(string url)
{
string result = string.Empty;
Uri Uris = new Uri(url);
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(Uris);
//req3.Proxy = proxy;
req.Method = "HEAD";
req.AllowAutoRedirect = false;
HttpWebResponse myResp = (HttpWebResponse)req.GetResponse();
if (myResp.StatusCode == HttpStatusCode.Redirect)
{
string temp = myResp.GetResponseHeader("Location");
//Recursive call
result = GetFinalRedirectedUrl(temp);
}
else
{
result = url;
}
return result;
}
Note: myResp.ResponseUri does not return the final URL