URL sometimes not being properly encoded - threading issue? - c#

I am working on a Windows Form application in C# and have a method like the following which is being accessed by multiple threads (precisely, by multiple background workers):
public Uri signURL(OAuthToken token, string url)
{
UriBuilder builder = new UriBuilder(addOAuthParameters(url));
NameValueCollection query = HttpUtility.ParseQueryString(builder.Query);
query.Set("oauth_consumer_key", consumerKey);
/*
* & sometimes not replaced by %26
*/
query.Set("oauth_signature", consumerSecret + "&" + token.Secret);
query.Set("oauth_token", token.Token);
builder.Query = query.ToString();
return builder.Uri;
}
I use this method to sign an arbitrary URL with some required OAuth parameters and afterwards do an HttpWebRequest to retrieve the content.
Edit 1: Here is the content of the addOAuthParameter method:
private Uri addOAuthParameters(string uri)
{
UriBuilder builder = new UriBuilder(uri);
NameValueCollection query = HttpUtility.ParseQueryString(builder.Query);
query.Set("oauth_signature_method", "PLAINTEXT");
query.Set("oauth_timestamp", "" + (int)(DateTime.UtcNow -
new DateTime(1970, 1, 1)).TotalSeconds);
query.Set("oauth_nonce", "" + getNonce());
query.Set("oauth_version", "1.0");
builder.Query = query.ToString();
return builder.Uri;
}
Sometimes, the oauth_signature parameter contains an ampersand although this should be properly encoded with %26 by the NameValueCollection object, and, as result, I get a "401 Unauthorized". I have the feeling this happens when the method is being accessed by multiple background workers (multiple threads?). Is that possible?
Edit 2: Okay, it seems that I've narrowed down the issue. If I do a Debug.Assert(builder.Uri.ToString().Contain("%26") && builder.Uri.PathAndQuery.Contains("%26")); it turns out that builder.Uri.ToString() does not contain the %26 while builder.Uri.PathAndQuery does. Now, why's that?
Debugging the issue turned out to be very hard. Does anyone have any suggestions?

Uri.ToString is for display only the docs state that it returns an un-escaped string.
You want to use Uri.AbsoluteUri, Uri.OriginalString or Uri.GetComponents.
var uri = new Uri("http://example.com/some?query=testing%26other");
Console.WriteLine(uri.ToString());
Console.WriteLine(uri.GetComponents(UriComponents.AbsoluteUri, UriFormat.UriEscaped));
http://example.com/some?query=testing&other
/some?query=testing%26other
GetComponents has the advantage that the escaping is explicitly specified so there's no ambiguity.
Some simple advice (originally from Keith Brown's blog post, Beware Uri.ToString).
Use Uri.AbsoluteUri to get the value of a URI when you know it’s absolute.
Use Uri.OriginalString to get the value of a URI when it could be either absolute or relative (this method does not throw an InvalidOperationException for a relative URI).
Use Uri.ToString to get the value of a URI only when you really want it to be unescaped (e.g. when you want to display it nicely for a human).
When viewing a URI in the debugger, remember the debugger uses Uri.ToString so what you see may not match exactly what the URI contains.

Related

Bug? System.UriFormatException: Invalid URI: The URI scheme is not valid

The strings are identical but when passed as a variable it is not valid?
What the hell is going on? Is it a language bug? I'm running this in C# .Net Core
var postUrl = "​http://www.contoso.com";
var postUri = new Uri("http://www.contoso.com"); // works
var uri = new Uri(postUrl); // does not work
If you pulling your hair, then it because there is space after first opening quote in postUrl. Please remove that space & your bug will be begone.
Worked around the problem by using.
var postUrl = "​http://www.contoso.com";
var uriBuilder = new UriBuilder(postUrl);
var uri = uriBuilder.Uri
Still wondering wtf?
Just the daily wtf of a programmer doing his job.

Uri.EscapeDataString or HttpUtility.UrlEncode does not work for my case

I have an API in a project which I coded as following:
[Route("api/SearchCustomer/{customer}")]
[HttpGet]
public List<Customer> SearchCustomer(string customer)
{
return customerRepo.GetSearchJoined(customer);
}
At first, I got an issue when I am calling this API from my front end, if customer contains dot or space(for example: https://www.somesite.com/walmart Inc.), I will get 404 error(cannot found this API). I find an easy way to solve this problem. Just add a "/" will solve this problem.(https://www.somesite.com/walmart Inc./ )
Now I need to call this API in another project at the back end. So I did something like this:
var urlParm = "https://www.somesite.com/api/SearchCustomer/" + name + "/";
response = client.GetAsync(urlParm).Result;
var dataObjects = response.IsSuccessStatusCode ? response.Content.ReadAsAsync<IList<Customer>>().Result : null;
return dataObjects;
Unfortunately, adding the "/" at back does not work. I am still getting 404 error. Then, I tried to use Uri.EscapeDataString or HttpUtility.UrlEncode to encode "name".(Does C# have an equivalent to JavaScript's encodeURIComponent()?)
name = Uri.EscapeDataString(name)
or name = HttpUtility.UrlEncode(name)
or name = HttpUtility.UrlPathEncode(name)
var urlParm = "https://www.somesite.com/api/SearchCustomer/" + name + "/";
or var urlParm = = "https://www.somesite.com/api/SearchCustomer/" + name
response = client.GetAsync(urlParm).Result;
var dataObjects = response.IsSuccessStatusCode ? response.Content.ReadAsAsync<IList<Customer>>().Result : null;
return dataObjects;
I have tried all the different matches of above code. All of them did not work. I am still getting the 404 error. Does anyone know what I am doing wrong here?
Sorry for the typo, I removed some sensitive information so I deleted the "api" by mistake. The route is not the problem. I have tested that the api call from the back end worksif name contains only letters or numbers but fails when name contains dot.
The problem is not relevant the customer parameter is encoded or not. You should specify the routing and apply the request correctly. Firstly fix the route;
[Route("api/SearchCustomer/{customer}")]
Then apply the request.
https://www.somesite.com/api/SearchCustomer/samplecustomer

How to extract first segment out of URI using Uri class

I am using Uri class for application development and needs the first segment of user-entered uri that either it contains http:// or http:// or ftp:// etc.
If not so,i have to hardcode to add to it.
I have already searched for it using googling and stackoverflowing but they didn't showed the precise requirement for me.
string path,downloadURL;
path = this.savePath.Text;
downloadURL = this.downloadURL.Text;
// i have done this but it didn't check if already existing .
downloadURL = "http://" + downloadURL;
Uri tmp = new Uri(downloadURL);
//extracts the last element
string EndPathFileName = tmp.Segments.Last();
// something like this but it returns only '/'.
//string StartPathFileName = tmp.Segments.First();
//Console.WriteLine(StartPathFileName);
Any suggestions?
Well there are a few options depending on what behavior you want...
You could just check if it contains :// which might be enough for what you want:
if(!downloadURL.Contains("://"))
downloadURL = "http://" + downloadURL;
Note that this would allow things such as "rubbish://www.example.com"
If you wanted to be a bit more cautious, you could check if the string starts with one of your predetermined values. For example:
if(!downloadURL.StartsWith("http://") && !downloadURL.StartsWith("https://") && !downloadURL.StartsWith("ftp://"))
downloadURL = "http://" + downloadURL;
Though this would mean that "rubbish://www.example.com" would become "http://rubbish://www.example.com".
You could go for a mix of both options, but keep in mind it can become very difficult to cope with all kinds of user input.
One final suggestion, which is even more robust, might be as follows:
string[] approvedSchemes = new string[] { "http", "https", "ftp" };
string userScheme = "";
if(downloadURL.Contains("://"))
{
// Get the first scheme defined, we will use this if it is in the approved list.
userScheme = downloadURL.Substring(0, downloadURL.IndexOf("://"));
// To cater for multiple :// remove all of them
downloadURL = downloadURL.Substring(downloadURL.LastIndexOf("://") + 3);
}
// Check if the user defined scheme is in the approved list, if not then set to http.
if(Array.IndexOf(approvedSchemes, userScheme.ToLowerInvariant()) > -1)
downloadURL = userScheme + "://" + downloadURL;
else
downloadURL = "http://" + downloadURL;
Here is a working example
You need to use Uri.Scheme Property
Uri baseUri = new Uri("http://www.contoso.com/");
Console.WriteLine(baseUri.Scheme); //http
Uri anotherUri = new Uri("https://www.contoso.com/");
Console.WriteLine(anotherUri.Scheme); //https

String that came from Request.Url.ToString() misteriously changes to another string when manipulating/comparing the first characters

I'm aware that there are easier ways to do this and believe me, I've tried them. I'm of course open to any suggestions =). You don't need to read the whole code, just the part that says where the problem lies. Also, I'm debbugging perl style so you guys can see. Oh and did I mention that on my development environment everything works as intended?
Here's the code:
string GetPortalAlias()
{
String myURL2 = Request.Url.ToString();
URLLabel.Text = "Original Request.Url.ToString() returned: \"" + myURL2 + "\"";
string myURL = string.Copy(myURL2);
URLLabel.Text = "Copying it to myURL, it's now: \"" + myURL + "\"";
myURL = myURL.ToLower().Trim();
URLLabel.Text += "<br>Trimming and ToLower myURL.<br>The new url is \"" + myURL + "\"" + "<br>";
myURL = myURL.Replace(":80", "");
URLLabel.Text += "Replacing the \":80\".<br> The new url is\"" + myURL + "\"<br>";
//***HERE LIES THE PROBLEM***
myURL = myURL.Replace("http://", "");
URLLabel.Text += "Replacing the \"http://\".<br> The new url is\"" + myURL + "\"<br>";
//***PROBLEM ENDS***
myURL = myURL.Remove(myURL.IndexOf("/"));
URLLabel.Text += "Removing everything after the \"/\"." + "<br> The new url is \"" + myURL + "\"<br>";
URLLabel.Text += "<br>GetPortalAlias Returning \"" + myURL + "\"";
return myURL;
}
Believe it or not, the output produced in the webpage is this:
Copying it to myURL, it's now: "http://sar.smg.com.ar/Default.aspx?TabID=912"
Trimming and ToLower myURL.
The new url is "http://sar.smg.com.ar/default.aspx?tabid=912"
Replacing the ":80".
The new url is"http://sar.smg.com.ar/default.aspx?tabid=912"
Replacing the "http://".
The new url is"intranetqa/default.aspx?tabid=912"
Removing everything after the "/".
The new url is "intranetqa"
GetPortalAlias Returning "intranetqa"
So... for some reason whenever it reaches the replace section it mysteriously mutates to start with "intranetqa" instead of "sar.smg.com.ar". "intranetqa" is our default hostname. CHANGING OR TAKING AWAY ANY CHARACTER OF HTTP:// IN ANY WAY MUTATES THE STRING.
I do a string.copy because I'm aware that if two strings are equal the compiler stores them in the same place therefore I wanted to prevent errors. Taking those lines away and use Request.Url.ToString() tomyURL directly does nothing at all. They were just a test to see if that worked.
Here's a list of the things I've tried:
All combinations of string / String, none worked.
I've tried Request.Host.Url and it just gave me "intranetqa".
I've used Request.Url.AbsoluteUri and that's why I have the replace
:80 line.
USING THE .tochararray FUNCTION GIVES ME BACK THE INTRANETQA THING
myURL = myURL.Substring(6) gives back the intranetqa thing.
string.Contains("sar.smg.com.ar") gives back false.
I believe the trick lies around here:
Uri uriAddress1 = Request.Url; and "The parts are <br>" + "Part 1: " + uriAddress1.Segments[0] + "<br>Part 2: " + uriAddress1.Segments[1]; Gives Part1 : "/" and Part 2: "Default.aspx". Trying to access part 3 (index 2) gives an exception.
The request.url does not have the first part, but when I call the ToString() method, it does have like a "fake" first part
Between your browser and the server are a reverse proxy and an output re-writer. These may be the same component, or separate components.
The URL your server actually sees is always of the form http://intranetqa/default.aspx?tabid=912 (after the reverse proxy/URL re-writer has intercepted the request).
The output your server produces is actually like:
Copying it to myURL, it's now: "http://intranetqa/Default.aspx?TabID=912"
Trimming and ToLower myURL.
The new url is "http://intranetqa/default.aspx?tabid=912"
Replacing the ":80".
The new url is"http://intranetqa/default.aspx?tabid=912"
Replacing the "http://".
The new url is"intranetqa/default.aspx?tabid=912"
Removing everything after the "/".
The new url is "intranetqa"
GetPortalAlias Returning "intranetqa"
The output re-writer is inspecting the output from your server and doing a replace of http://intranetqa with http://sar.smg.com.ar. Once you strip the http:// off of the front of these strings, it's no longer a match and so replacement no longer occurs.
If you want to know what the original requesting URL/host are, hopefully the reverse proxy either is, or can be configured to, adding an extra header to the request with the original URL.
You can try something like this
Uri uriAddress1 = new Uri("http://www.contoso.com/title/index.htm");
Console.WriteLine("The parts are {0}, {1}, {2}", uriAddress1.Segments[0], uriAddress1.Segments[1], uriAddress1.Segments[2]);
Uri.Segments Property
This is better way to handle URIs and their segments.
Try to use this property instead:
String myURL2 = Request.Url.AbsoluteUri;
Here is an Extension method that I use to pull the SiteRootPath. You should be able to easily adjust it however you need it. You will need access to the HttpContext for what I currently have below, however, you don't sound like you need that.
using System;
using System.Web;
namespace FlixPicks.Web.Extensions
{
public static class HttpContextExtensions
{
public static string SiteRootPath(this HttpContext context)
{
if (context == null || context.Request == null) { return null; }
return context.Request.Url.SiteRootPath(context.Request.ApplicationPath);
}
public static string SiteRootPath(this HttpContextBase context)
{
return context.Request.Url.SiteRootPath(context.Request.ApplicationPath);
}
private static string SiteRootPath(this Uri url, string applicationPath)
{
if (url == null) { return null; }
// Formatting the fully qualified website url/name.
string appPath = string.Format(
"{0}://{1}{2}{3}",
url.Scheme,
url.Host,
url.Port == 80 ? string.Empty : ":" + url.Port,
applicationPath);
// Remove ending slash(es) if one or more exists to consistently return
// a path without an ending slash. Could have just as well choosen to always include an ending slash.
while (appPath.EndsWith("/") || appPath.EndsWith("\\"))
{
appPath = appPath.Substring(0, appPath.Length - 1);
}
return appPath;
}
}
}
Good luck,
Tom
Don't you want to achieve part of what is done here?
Something like
string host = Request.Url.IsDefaultPort ?
Request.Url.Host :
Request.Url.Authority;
If you want to persist with the old method change it like this
string GetPortalAlias()
{
var rawUrl = Request.Url.ToString();
var lowerTrimmedUrl = rawUrl.ToLower().Trim();
var withoutPortUrl = lowerTrimmedUrl.Replace(":80", "");
var withoutProtocolUrl = withoutPortUrl.Replace("http://", "");
var justHostUrl = withoutProtocolUrl.Remove(myURL.IndexOf("/"));
var evolution = new StringBuilder();
evolution.AppendFormat(
"{0}<br>",
HttpUtility.HtmlEncode(rawUrl));
evolution.AppendFormat(
"{0}<br>",
HttpUtility.HtmlEncode(lowerTrimmedUrl));
evolution.AppendFormat(
"{0}<br>",
HttpUtility.HtmlEncode(withoutPortUrl));
evolution.AppendFormat(
"{0}<br>",
HttpUtility.HtmlEncode(withoutProtocolUrl));
evolution.AppendFormat(
"{0}<br>",
HttpUtility.HtmlEncode(justHostUrl));
URLLabel.Text = evolution.ToString();
return justHostUrl;
}
So you can see whats going on.

What is the quickest way to get the absolute uri for the root of the app in asp.net?

What is the simplest way to get: http://www.[Domain].com in asp.net?
There doesn't seem to be one method which can do this, the only way I know is to do some string acrobatics on server variables or Request.Url. Anyone?
We can use Uri and his baseUri constructor :
new Uri(this.Request.Url, "/") for the root of the website
new Uri(this.Request.Url, this.Request.ResolveUrl("~/")) for the root of the website
You can do it like this:
string.Format("{0}://{1}:{2}", Request.Url.Scheme, Request.Url.Host, Request.Url.Port)
And you'll get the generic URI syntax <protocol>://<host>:<port>
You can use something like this.
System.Web.HttpContext.Current.Server.ResolveUrl("~/")
It maps to the root of the application. now if you are inside of a virtual directory you will need to do a bit more work.
Edit
Old posting contained incorrect method call!
I really like the way CMS handled this question the best, using the String.Format, and the Page.Request variables. I'd just like to tweak it slightly. I just tested it on one of my pages, so, i'll copy the code here:
String baseURL = string.Format(
(Request.Url.Port != 80) ? "{0}://{1}:{2}" : "{0}://{1}",
Request.Url.Scheme,
Request.Url.Host,
Request.Url.Port)
System.Web.UI.Page.Request.Url
this.Request.Url.Host
I use this property on Page to handle cases virtual directories and default ports:
string FullApplicationPath {
get {
StringBuilder sb = new StringBuilder();
sb.AppendFormat("{0}://{1}", Request.Url.Scheme, Request.Url.Host);
if (!Request.Url.IsDefaultPort)
sb.AppendFormat(":{0}", Request.Url.Port);
if (!string.Equals("/", Request.ApplicationPath))
sb.Append(Request.ApplicationPath);
return sb.ToString();
}
}
This method handles http/https, port numbers and query strings.
'Returns current page URL
Function fullurl() As String
Dim strProtocol, strHost, strPort, strurl, strQueryString As String
strProtocol = Request.ServerVariables("HTTPS")
strPort = Request.ServerVariables("SERVER_PORT")
strHost = Request.ServerVariables("SERVER_NAME")
strurl = Request.ServerVariables("url")
strQueryString = Request.ServerVariables("QUERY_STRING")
If strProtocol = "off" Then
strProtocol = "http://"
Else
strProtocol = "https://"
End If
If strPort <> "80" Then
strPort = ":" & strPort
Else
strPort = ""
End If
If strQueryString.Length > 0 Then
strQueryString = "?" & strQueryString
End If
Return strProtocol & strHost & strPort & strurl & strQueryString
End Function
I had to deal with something similar, I needed a way to programatically set the tag to point to my website root.
The accepted solution wasn't working for me because of localhost and virtual directories stuff.
So I came up with the following solution, it works on localhost with or without virtual directories and of course under IIS Websites.
string.Format("{0}://{1}:{2}{3}", Request.Url.Scheme, Request.Url.Host, Request.Url.Port, ResolveUrl("~")
Combining the best of what I've seen on this question so far, this one takes care of:
http and https
standard ports (80, 443) and non standard
application hosted in a sub-folder of the root
string url = String.Format(
Request.Url.IsDefaultPort ? "{0}://{1}{3}" : "{0}://{1}:{2}{3}",
Request.Url.Scheme, Request.Url.Host,
Request.Url.Port, ResolveUrl("~/"));

Categories