I need to get the url of the final destination of a shortened url. At the moment I am doing the following which seems to work:
var request = WebRequest.Create(shortenedUri);
var response = request.GetResponse();
return response.ResponseUri;
But can anyone suggest a better way?
If this shortened url is generated by some online service provider it is only this service provider that is storing the mapping between the short and the actual url. So you need to query this provider by sending an HTTP request to it, exactly as you did. Also don't forget to properly dispose IDisposable resources by wrapping them in using statements:
var request = WebRequest.Create(shortenedUri);
using (var response = request.GetResponse())
{
return response.ResponseUri;
}
If the service provider supports the HEAD verb you could also use this verb and read the Location response HTTP header which must be pointing to the actual url. As an alternative you could set the AllowAutoRedirect property to false on the request object and then read the Location response HTTP header. This way the client won't be redirecting to the actual resource and getting the entire response body when you are not interested in it.
Of course the best way to do this would be if your online service provider offers an API that would allow you to directly give you the actual url from a short url.
You do need to make an HTTP request - but you don't need to follow the redirect, which WebRequest will do by default. Here's a short example of making just one request:
using System;
using System.Net;
class Test
{
static void Main()
{
string url = "http://tinyurl.com/so-hints";
Console.WriteLine(LengthenUrl(url));
}
static string LengthenUrl(string url)
{
var request = WebRequest.CreateHttp(url);
request.AllowAutoRedirect = false;
using (var response = request.GetResponse())
{
var status = ((HttpWebResponse) response).StatusCode;
if (status == HttpStatusCode.Moved ||
status == HttpStatusCode.MovedPermanently)
{
return response.Headers["Location"];
}
// TODO: Work out a better exception
throw new Exception("No redirect required.");
}
}
}
Note that this means if the "lengthened" URL is itself a redirect, you won't get the "final" URI as you would in your original code. Likewise if the lengthened URL is invalid, you won't spot that - you'll just get the URL that you would have redirected to. Whether that's a good thing or not depends on your use case...
Related
I'm attempting to pass username/password from an application to the API to receive a token authorization key. When I attempt to do so, I receive a 400 Bad Request error and I cannot figure out why. Below is the method in question:
public User UserAuthentication(string username, string password)
{
string endpoint = baseURL + "/TOKEN";
// Could be POST maybe
string method = "POST";
Credential jsonObj = new Credential
{
grant_type = "password",
username = username,
password = password
};
string jsonStr = JsonConvert.SerializeObject(jsonObj);
WebClient wc = new WebClient();
//x - www - form - urlencoded
wc.Headers[HttpRequestHeader.ContentType] = "application/x - www - form - urlencoded";
wc.Headers.Add("Access-Control-Allow-Headers", "content-type");
wc.Headers.Add("Access-Control-Allow-Origin", "*");
wc.Headers[HttpRequestHeader.Authorization] = "Bearer <token>";
wc.Headers.Add("Access-Control-Allow-Methods", "POST, PUT, GET, DELETE, OPTIONS");
string header = wc.Headers.ToString();
try
{
string response = wc.UploadString(endpoint, method, jsonStr);
return JsonConvert.DeserializeObject<User>(response);
}
catch(Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
I've messed around altering just about everything in this method in search of a fix.
What I've done:
/TOKEN was /values & /api/values
POST method was GET -- With this, I received a "Cannot send a content-body with this verb-type." error.
ContentType was changed to "application/json"
Access-Control-Allow-Origin had the baseURL
Checked the format of header & body:
Header:
{Content-Type: application/x - www - form - urlencoded
Access-Control-Allow-Headers: content-type
Access-Control-Allow-Origin: *
Authorization: Bearer <token>
Access-Control-Allow-Methods: POST, PUT, GET, DELETE, OPTIONS}
Body:
{"grant_type":"password",
"username":"test#gmail.com",
"password":"password123"}
I obviously have something wrong in my request, I've just run out of ideas to try. I'm not entirely sure if UploadString() is the correct method to be using in this situation, but I couldn't find another method in the WebClient class that would be better. Any help to try and push me in the right direction would be very much appreciated.
So what I think you are trying to do is a form-urlencoded post to a "token" endpoint with a username/password grant. These are typically done like so:
using (var request = new HttpRequestMessage(HttpMethod.Post, new Uri("https://example.com/token"))
{
Content = new FormUrlEncodedContent(new Dictionary<string, string>
{
{ "grant_type", "password" },
{ "username", "username#site.com" },
{ "password", "password12345" }
})
})
{
using (var resp = await _client.SendAsync(request))
{
resp.EnsureSuccessStatusCode();
//await resp.Content.ReadAsAsync<BearerToken>();
// for testing purposes, try this:
var returnData = await resp.Content.ReadAsStringAsync();
Console.WriteLine(returnData);
}
}
You should define this outside all scopes where you need to do Http requests:
private static readonly HttpClient _client = new HttpClient();
So, first off, try to stick with HttpClient. Other patterns such as WebClient are considered legacy.
Next, CORS headers are typically returned from the server when an OPTIONS call is sent to the server. You aren't doing that here, and you should never have to worry about that kind of stuff inside a C# program running from your computer. So you can drop the access-control header stuff.
Form-urlencoded data is not JSON data. It's a different way to format data. If you want to send JSON data, you should use the content-type application/json
Finally, you are trying to add an Authorization header. But that doesn't make much sense as you are trying to authenticate yourself to become authorized. If you send the right username/password, you will receive a bearer token that you can use in an Authorization header for future requests to said service.
Oh and I forgot to add: Whenever you see an error in the [400,499] range (in this case "400 - bad request") it means that you sent something wrong and the server doesn't understand what you are trying to do. For example: a 401 means you sent invalid or missing authorization information. A 400 means your data was probably malformed.
But I like your question... I can see what you were doing and you tried all kinds of different things.
Download a program called Fiddler if you want to see how HTTP works. It's a great tool to debug your HTTP calls.
I am struggling with Rest call. Here is my code and it is working for basic authentication.
public async Task RunAsync(string name, string value)
{
using (var handler = new HttpClientHandler { UseDefaultCredentials = true })
using (var client = new HttpClient(handler))
{
var byteArray = Encoding.ASCII.GetBytes("username:password");
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", Convert.ToBase64String(byteArray));
var urlRestGet = HomeController.url;
client.BaseAddress = new Uri(urlRestGet + "?name=" + name + "&value=" + value + "");
client.DefaultRequestHeaders.Accept.Clear();
**1. if(HomeController.contentType.ToLower()=="xml"){
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/xml"));
}**
else if (HomeController.contentType.ToLower() == "json")
{
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
}
HttpResponseMessage response = await client.GetAsync(urlRestGet + "?name=" + name + "&value=" + value + "");
if (response.IsSuccessStatusCode)
{
//Get the response
loginJsonString = await response.Content.ReadAsStringAsync();
//Converting to xml
using (var stream = new MemoryStream(Encoding.ASCII.GetBytes(loginJsonString)))
{
var output = new XmlDictionaryReaderQuotas();
xmlResult = XDocument.Load(JsonReaderWriterFactory.CreateJsonReader(stream, output)).ToString();
}
}
}
}
1) If the content type is application/xml am I correct to use line 1 part in the code.
2) How can I make this code more generic. (when the authentication type is different eg: tokenized or cookiebased how can I change this.)
There are a couple of things about your code I do not understand.
What is HomeController.contentType all about? The name "HomeController" suggests you're writing an ASP.NET MVC Controller (serverside). Though you seem to be writing something intended to be used as a HTTP client. But I could be mistaken or mislead here.
You are reading a Json response, then loading it as a Xml document?
I'll try to answer anyway.
1) If the content type is application/xml am I correct to use line 1 part in the code.
The Accept header sent by the client (you) tells the server that you accept the given content type. If you send Accept application/xml you tell the server you prefer if the response is Xml.
Your code seem to assume the response's content type is always Json.
You could include both application/xml and application/json as Accept headers in your request. The server should honor that request and pick the first supported content type for it's response.
When processing the response you should check the actual content type and handle the response content appropriately.
Remember that Accept only tells the server that you prefer those content types. The server may decide not to honor your whishes and can return any content type it desires.
2) How can I make this code more generic. (when the authentication type is different eg: tokenized or cookiebased how can I change this.)
If you mean tokenized as in a query parameter you should probably handle your query parameters as a collection rather than a hardcoded formatted string.
Check out the NameValueCollection class and this SO question on NameValueCollection to query string.
To handle cookies, you basically need to copy/re-use the cookie collection returned in a response in the next request.
See this SO question on how to inject cookies when you create a new HttpClient.
... but it's much easier to use a library
As you already discovered, making a robust REST/HTTP client is not a easy task.
And as #Mafii and #Fildor already pointed out in comments there are already numerous libraries available. RestSharp (https://restsharp.org) being one very popular.
I have a proxy controller in order to redirect the Ajax requests and pass the same cookies from the current domain to the Web API endpoint, but it doesn't work as what I expected. e.g. the cookies in "https://www.example.com", the Web API URL "https://api.example.com/xyz/abc/". What I am trying to do is sending an Ajax request to
"https://www.example.com/api/proxy/something"
and hoping it to be redirected to
"https://api.example.com/xyz/abc/something" with the same settings (especially the cookies).
Here is the API controller in the web site:
public class ProxyController : ApiController
{
private string _baseUri = "https://api.example.com/xyz/abc/";
[AcceptVerbs(Http.Get, Http.Head, Http.MkCol, Http.Post, Http.Put)]
public async Task<HttpResponseMessage> Proxy()
{
using (HttpClient http = new HttpClient())
{
string proxyURL = this.Request.RequestUri.AbsolutePath;
int indexOfProxy = proxyURL.IndexOf("proxy/") + 6;
_baseUri = _baseUri + proxyURL.Substring(indexOfProxy, proxyURL.Length - indexOfProxy);
this.Request.RequestUri = new Uri(_baseUri);
//For some reason Ajax request sets Content in the Get requests, because
//of that it was denied complaining about "cannot send a content-body"
if (this.Request.Method == HttpMethod.Get)
{
this.Request.Content = null;
}
return await http.SendAsync(this.Request);
}
}
}
It doesn't redirect the requests. In the response, the Requested URL is the same as the original request. The Host in the request header is "www.example.com" instead of "api.example.com', I am going nuts with this issue for the last few days.
HttpClient will work fine. We have done this many times. However, cookies are tricky as they are tied to the domain. Here is some code of a proxy without cookies that should get you started.
[AcceptVerbs(Http.Get, Http.Head, Http.MkCol, Http.Post, Http.Put)]
public async Task<HttpResponseMessage> Proxy()
{
var request = this.Request;
var proxyUri = this.GetProxyUri(request.RequestUri);
request.RequestUri = proxyUri;
request.Headers.Host = proxyUri.Host;
if (request.Method == HttpMethod.Get)
{
request.Content = null;
}
//todo: Clone all cookies with the domain set to the domain of the proxyUri. Remove the old cookies and add the clones.
using (var client = new HttpClient())
{
//default is 60 seconds or so
client.Timeout = TimeSpan.FromMinutes(5);
return await client.SendAsync(request, HttpCompletionOption.ResponseContentRead);
}
}
private string _baseUri = "https://api.example.com/xyz/abc/";
private Uri GetProxyUri(Uri originalUri)
{
var proxyUri = originalUri.AbsolutePath;
var indexOfProxy = proxyUri.IndexOf("proxy/") + 6;
var finalUri = _baseUri + proxyUri.Substring(indexOfProxy, proxyUri.Length - indexOfProxy);
return new Uri(finalUri);
}
You may not be able to get cookies to work properly because of the domain switching. Your client and server will be limited to it's domain. If you own the proxy destination, you may want to change it to allow other mechanisms besides cookies. Can you use headers, or querystring, etc? The domain just may be a killer.
I had exactly this kind of problem like two days ago.
Haven't figured out exactly what was causing it, I'm going to investigate today and let you know.
But until then, you can try using RestSharp, it fixed my problem.
I think it is a problem with the BaseAddress property of the HttpClient, it somehow gets initialized with the initial address and makes the request to that address instead of the proxy one.
I will investigate and let you know.
I have a function that is coded to get the Content-Type of a web file.
Here is the function:
public string GetContentTypeOfUri(string uri)
{
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "HEAD";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
var contentType = response.Headers["Content-Type"];
return (contentType);
}
}
catch (Exception ex)
{
return "error";
}
}
Rather than writing a whole different function to detect if a web file exists, how can I calculate if a file exists from the same code as used to get the Content-Type?
If I use a uri of a file that does not exist, an exception occurs. The ex.HResult equals -2146233079 when this exception occurs, with a message = "The remote name could not be resolved: '[address name]'".
Is it safe to say that when an exception occurs, and the ex.HResult equals -2146233079, the file does not exist?
Is there an easier/better way to work this out?
Thanks in advance
EDIT
Here is the HttpClient code that I have:
public async Task<string> GetContentTypeAsync(string uri)
{
using (var httpClient = new HttpClient())
{
var request = new HttpRequestMessage(HttpMethod.Post, uri);
request.Method = new HttpMethod("HEAD");
var response = await httpClient.SendAsync(request);
string contentType = response.Content.Headers.ContentType.ToString();
return contentType;
}
}
Your example web address does inform me that the address does not exist, however, if I have a web address that does not exist such as http://www.usa.canon.com/app/html/HDV/HG10/images/hg10_sample_image_03.jpg5, I am getting a StatusCode of OK, as a Text content type is returned as a custom error page.
There are two possible "not exists" scenarios. It sounds like you've identified one of them - when the server name in the URL is incorrect and so the request cannot even be sent.
But you're not accounting for the other error - that you can reach a remote server but it denies all knowledge of a specific file. For that scenario, you ought to be checking for status 404 on the response.
For cleaner handling of your current scenario (server doesn't exist) you could use the Uri class to extract the Host name from the uri string and perform a manual DNS lookup - which would allow you to code for this likely scenario without having to catch exceptions - which is generally frowned upon when its an expected scenario.
I created RESTful webservice (WCF) where I check credentials on each request. One of my clients is Android app and everything seems to be great on server side. I get request and if it's got proper header - I process it, etc..
Now I created client app that uses this service. This is how I do GET:
// Create the web request
var request = WebRequest.Create(Context.ServiceURL + uri) as HttpWebRequest;
if (request != null)
{
request.ContentType = "application/json";
// Add authentication to request
request.Credentials = new NetworkCredential(Context.UserName, Context.Password);
// Get response
using (var response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
if (response != null)
{
var reader = new StreamReader(response.GetResponseStream());
// Console application output
var s = reader.ReadToEnd();
var serializer = new JavaScriptSerializer();
var returnValue = (T)serializer.Deserialize(s, typeof(T));
return returnValue;
}
}
}
So, this code get's my resource and deserializes it. As you see - I'm passing credentials in my call.
Then when debugging on server-side I noticed that I get 2 requests every time - one without authentication header and then server sends back response and second request comes bach with credentials. I think it's bad for my server - I'd rather don't make any roundtrips. How should I change client so it doesn't happen? See screenshot of Fiddler
EDIT:
This is JAVA code I use from Android - it doesn't do double-call:
MyHttpResponse response = new MyHttpResponse();
HttpClient client = mMyApplication.getHttpClient();
try
{
HttpGet request = new HttpGet(serviceURL + url);
request.setHeader(new BasicHeader(HTTP.CONTENT_TYPE, "application/json"));
request.addHeader("Authorization", "Basic " + Preferences.getAuthorizationTicket(mContext));
ResponseHandler<String> handler = new BasicResponseHandler();
response.Body = client.execute(request, handler);
response.Code = HttpURLConnection.HTTP_OK;
response.Message = "OK";
}
catch (HttpResponseException e)
{
response.Code = e.getStatusCode();
response.Message = e.getMessage();
LogData.InsertError(mContext, e);
}
The initial request doesn't ever specify the basic header for authentication. Additionally, since a realm is specified, you have to get that from the server. So you have to ask once: "hey, I need this stuff" and the server goes "who are you? the realm of answering is 'secure area'." (because realm means something here) Just because you added it here:
request.Credentials = new NetworkCredential(Context.UserName, Context.Password);
doesn't mean that it's going to be for sure attached everytime to the request.
Then you respond with the username/password (in this case you're doing BASIC so it's base64 encoded as name:password) and the server decodes it and says "ok, you're all clear, here's your data".
This is going to happen on a regular basis, and there's not a lot you can do about it. I would suggest that you also turn on HTTPS since the authentication is happening in plain text over the internet. (actually what you show seems to be over the intranet, but if you do go over the internet make it https).
Here's a link to Wikipedia that might help you further: http://en.wikipedia.org/wiki/Basic_access_authentication
Ok, I got it. I manually set HttpHeader instead of using request.Credentials
request.Headers.Add(HttpRequestHeader.Authorization, "Basic " + Convert.ToBase64String(Encoding.UTF8.GetBytes(Context.UserName + ":" + Context.Password)));
Now I see only single requests as expected..
As an option you can use PreAuthenticate property of HttpClientHandler. This would require a couple of lines more
var client = new HttpClient(new HttpClientHandler
{
Credentials = yourCredentials,
PreAuthenticate = true
});
With using this approach, only the first request is sent without credentials, but all the rest requests are OK.