i'm trying to encode an url with the code below;
var encodedUrl = HttpUtility.UrlEncode("http://www.example.com");
var decodedUrl = HttpUtility.UrlDecode("http%3A%2F%2Fwww%2Eexample%2Ecom%2F");
I'm working with the google webmaster tools api and this api expects an URL as shown in the decodedUrl variable above. Every single character is encoded there.
When i use the httputility encode function i get the following result;
http%3a%2f%2fwww.example.com
How can i use the encoding variable in such a way that every character in the url is encoded?
I'm pretty sure that HtmlUtility and AntiXss (another MS tool for encoding urls) aren't going to help here. A "." in a url is considered valid and so doesn't need to be encoded.
I think you're going to have to post-process your encoded string to further encode other characters that are not valid within teh google webmaster tools API.
i.e. do something like this...
var encodedUrl = HttpUtility.UrlEncode("http://www.example.com")
.Replace(".", "%2E");
... assuming that "." is the only character you're having problems with.
The period is not a reserved character in a URL, so it won't be encoded. See this question and answer for an elegant solution.
Related
I'm trying to UrlEncode a web address using Uri.EscapeDataString, but the result isn't correct. Here's an example:
string url = "https://mega.co.nz/#!GVZFwAbB!NzdN2jp7A_WmQBLC4RJrCX8SzixFIEo7oZZARaMAmXQ";
string encodedUrl = Uri.EscapeDataString(url);
Expected result would be:
https%3a%2f%2fmega.co.nz%2f%23!GVZFwAbB!NzdN2jp7A_WmQBLC4RJrCX8SzixFIEo7oZZARaMAmXQ
But the actual one is:
https%253a%252f%252fmega.co.nz%252f%2523%21GVZFwAbB%21NzdN2jp7A_WmQBLC4RJrCX8SzixFIEo7oZZARaMAmXQ
As you can see, there's a bunch of extra %25s that don't belong there. Isn't %25 the encode for "%"? There are no %s in my original string... what's going on?
EDIT: I can't use the System.Web assembly for this project, so unfortunately I can't use the HttpUtility.UrlEncode() method for this.
Well, after searching around a bit more, it seems that this does the job, without relying on system web:
System.Net.WebUtility.UrlEncode(url);
The encoding is the correct one, without %25s.
Uri.EscapeDataString doesn't encode URL. Use HttpUtility.UrlEncode instead.
string url = "https://mega.co.nz/#!GVZFwAbB!NzdN2jp7A_WmQBLC4RJrCX8SzixFIEo7oZZARaMAmXQ";
string encodedUrl = HttpUtility.UrlEncode(url);
Result is:
https%3a%2f%2fmega.co.nz%2f%23!GVZFwAbB!NzdN2jp7A_WmQBLC4RJrCX8SzixFIEo7oZZARaMAmXQ
I have a WP7 project where I am using the below code. It normally works ok, but I am getting a strange result with some particular strings being passed through.
Service = "3q%23L3t41tGfXQDTaZMbn%23w%3D%3D?f"
NavigationService.Navigate(new Uri("/Details.xaml?service=" + Service, UriKind.Relative));
Next Page:
NavigationContext.QueryString.TryGetValue("service", out Service1);
Service1 now = 3q#L3t41tGfXQDTaZMbn#w==?f
Why has the string changed?
The string hasn't changed, but you're looking at it in two different ways.
The way to encode 3q#L3t41tGfXQDTaZMbn#w==?f for as URI content is as 3q%23L3t41tGfXQDTaZMbn%23w%3D%3D?f. (Actually, it's 3q%23L3t41tGfXQDTaZMbn%23w%3D%3D%3Ff but you get away with the ? near the end not being properly escaped to %3F in this context).
Your means of writing the string, expects to receive it escaped.
Your means of reading the string, returns it unescaped.
Things are working pretty much perfectly, really.
When you need to write the string again, then just escape it again:
Service = Uri.EscapeDataString(Service1);
In your first code snippet the string is URL Encoded.
In the 2nd code snippet, the string is URL Decoded.
They are essentially the same strings, just with encoding applied/removed.
For example: urlencoding # you get %23
For further reading check out this wikipedia article on encoding.
Since HttpUtility isn't part of WP7 Silverlight stack, I'd recommend using Uri.EscapeUriString to escape any URI's that have not been escaped.
You should probably URL encode the string if you want it to pass through unscathed.
I am trying to stop XSS attack so I am using html agility pack to make my whitelist and Microsoft Anti-Cross Site Scripting Library to deal with the rest.
Now I am looking at encoding all html hrefs. I get a big string of html code that can contain hrefs. Accours to MS Library they have an URL encode but if you encode the whole URl then it can't be used. So in the example they just encode the query string
UrlEncode Untrusted input is used in a
URL (such as a value in a
querystring) Click
Here!
http://msdn.microsoft.com/en-us/library/aa973813.aspx
So now my questions is how do I parse through a href and find the query string. Is it always just "?" then query string or can it have spaces and be written in different ways?
Edit
This urls will not be written by me but the users who will share them. So that's why I need a way to make sure I get all query strings and not just ones in valid format. If it can work invalid format I have to grab these ones too. Hackers won't care if it is valid format or not as long as it still does what they want.
I believe it is always the part after the ? but you can easily use the Uri class for this:
Uri uri = new Uri("http://foo.com/page.html?query");
string query = uri.Query;
That will include the ? itself. Of course, you can fetch the other bits as well, which could be handy.
what about using encrypted query string and in your code you can decrypt it
OR you can use Request.PathInfo that make you not need ? in query string
Here's a W3C reference addressing the composition of URIs with querystrings, which says in part:
The question mark ("?", ASCII 3F hex)
is used to delimit the boundary
between the URI of a queryable object,
and a set of words used to express a
query on that object.
In my web app, my parameters can contain all sorts of crazy characters (russian chars, slashes, spaces etc) and can therefor not always be represented as-is in a URL.
Sending them on their merry way will work in about 50% of the cases. Some things like spaces are already encoded somewhere (I'm guessing in the Html.BuildUrlFromExpression does). Other things though (like "/" and "*") are not.
Now I don't know what to do anymore because if I encode them myself, my encoding will get partially encoded again and end up wrong. If I don't encode them, some characters will not get through.
What I did is manually .replace() the characters I had problems with.
This is off course not a good idea.
Ideas?
--Edit--
I know there are a multitude of encoding/decoding libraries at my disposal.
It just looks like the mvc framework is already trying to do it for me, but not completely.
<a href="<%=Html.BuildUrlFromExpression<SearchController>(c=>c.Search("", 1, "a \v/&irdStr*ng"))%>" title="my hat's awesome!">
will render me
<a href="/Search.mvc/en/Search/1/a%20%5Cv/&irdStr*ng" title="my hat's awesome!">
Notice how the forward slash, asterisk and ampersand are not escaped.
Why are some escaped and others not? How can I now escape this properly?
Am I doing something wrong or is it the framework?
Parameters should be escaped using Uri.EscapeDataString:
string url = string.Format("http://www.foo.bar/page?name={0}&address={1}",
Uri.EscapeDataString("adlknad /?? lkm#"),
Uri.EscapeDataString(" qeio103 8182"));
Console.WriteLine(url);
Uri uri = new Uri(url);
string[] options = uri.Query.Split('?','&');
foreach (string option in options)
{
string[] parts = option.Split('=');
if (parts.Length == 2)
{
Console.WriteLine("{0} = {1}",parts[0],
Uri.UnescapeDataString(parts[1]));
}
}
AS others have mentioned, if you encode your string first you aviod the issue.
The MVC Framework is encoding characters that it knows it needs to encode, but leaving those that are valid URL characters (e.g. & % ? * /). This is because these are valid URL characters, although they are special chracters in a URL that might not acheive the result you are after.
Try using the Microsoft Anti-Cross Site Scripting library. It contains several Encode methods, which encode all the characters (including #, and characters in other languages). As for decoding, the browser should handle the encoded Url just fine, however if you need to manually decode the Url, use Uri.UnescapeDataString
Hope that helps.
Escaping of forward slahes and dots in path part of url is prohibited by security reason (althrough, it works in mono).
Html.BuildUrlFromExpression needs to be fixed then, would submit this upstream to the MVC project... alternatively do the encoding to the string before passing to BuildUrlFromExpression, and decode it when it comes back out on the other side.
It may not be readily fixable, as IIS may be handling the decoding of the url string beforehand... may need to do some more advanced encoding/decoding for alternative path characters in the utility methods, and decode on your behalf coming out.
I've seen similar posts on this. Too me, it looks like a flaw in MVC. The function would be more appropriately named "BuildUrlFromEncodedExpression". Whats worse, is that the called function needs to decode its input parameters. Yuk.
If there is any overlap between the characters encoded BuildUrlFromExpression() and the characters encoded by the caller (who, I think might fairly just encode any non-alphanumeric for simplicities sake) then you have potential for nasty bugs.
Server.URLEncode or HttpServerUtility.UrlEncode
I see what you're saying now - I didn't realize the question was specific to MVC. Looks like a limitation of that part of the MVC framework - particularly BuildUrlFromExpression is doing some URL encoding, but it knows that also needs some of those punctation as part of the framework URLs.
And also unfortunately, URLEncoding doesn't produce an invariant, i.e.
URLEncode(x) != URLEncode(URLEncode(x))
Wouldn't that be nice. Then you could pre-encode your variables and they wouldn't be double encoded.
There's probably an ASP.NET MVC framework best practice for this. I guess another thing you could do is encode into base64 or something that is URLEncode-invariant.
Have you tried using the Server.UrlEncode() method to do the encoding, and the Server.UrlDecode() method to decode?
I have not had any issues with using it for passing items.
I'm trying to pass in a Base64 string into a C#.Net web application via the QueryString. When the string arrives the "+" (plus) sign is being replaced by a space. It appears that the automatic URLDecode process is doing this. I have no control over what is being passed via the QueryString. Is there any way to handle this server side?
Example:
http://localhost:3399/Base64.aspx?VLTrap=VkxUcmFwIHNldCB0byAiRkRTQT8+PE0iIHBsdXMgb3IgbWludXMgNSBwZXJjZW50Lg==
Produces:
VkxUcmFwIHNldCB0byAiRkRTQT8 PE0iIHBsdXMgb3IgbWludXMgNSBwZXJjZW50Lg==
People have suggested URLEncoding the querystring:
System.Web.HttpUtility.UrlEncode(yourString)
I can't do that as I have no control over the calling routine (which is working fine with other languages).
There was also the suggestion of replacing spaces with a plus sign:
Request.QueryString["VLTrap"].Replace(" ", "+");
I had though of this but my concern with it, and I should have mentioned this to start, is that I don't know what other characters might be malformed in addition to the plus sign.
My main goal is to intercept the QueryString before it is run through the decoder.
To this end I tried looking at Request.QueryString.toString() but this contained the same malformed information. Is there any way to look at the raw QueryString before it is URLDecoded?
After further testing it appears that .Net expects everything coming in from the QuerString to be URL encoded but the browser does not automatically URL encode GET requests.
The suggested solution:
Request.QueryString["VLTrap"].Replace(" ", "+");
Should work just fine. As for your concern:
I had though of this but my concern with it, and I should have mentioned this to start, is that I don't know what other characters might be malformed in addition to the plus sign.
This is easy to alleviate by reading about base64. The only non alphanumeric characters that are legal in modern base64 are "/", "+" and "=" (which is only used for padding).
Of those, "+" is the only one that has special meaning as an escaped representation in URLs. While the other two have special meaning in URLs (path delimiter and query string separator), they shouldn't pose a problem.
So I think you should be OK.
You could manually replace the value (argument.Replace(' ', '+')) or consult the HttpRequest.ServerVariables["QUERY_STRING"] (even better the HttpRequest.Url.Query) and parse it yourself.
You should however try to solve the problem where the URL is given; a plus sign needs to get encoded as "%2B" in the URL because a plus otherwise represents a space.
If you don't control the inbound URLs, the first option would be preferred as you avoid the most errors this way.
I'm having this exact same issue except I have control over my URL. Even with Server.URLDecode and Server.URLEncode it doesn't convert it back to a + sign, even though my query string looks as follows:
http://localhost/childapp/default.aspx?TokenID=0XU%2fKUTLau%2bnSWR7%2b5Z7DbZrhKZMyeqStyTPonw1OdI%3d
When I perform the following.
string tokenID = Server.UrlDecode(Request.QueryString["TokenID"]);
it still does not convert the %2b back into a + sign. Instead I have to do the following:
string tokenID = Server.UrlDecode(Request.QueryString["TokenID"]);
tokenID = tokenID.Replace(" ", "+");
Then it works correctly. Really odd.
I had similar problem with a parameter that contains Base64 value and when it comes with '+'.
Only Request.QueryString["VLTrap"].Replace(" ", "+"); worked fine for me;
no UrlEncode or other encoding helping because even if you show encoded link on page yourself with '+' encoded as a '%2b' then it's browser that changes it to '+' at first when it showen and when you click it then browser changes it to empty space. So no way to control it as original poster says even if you show links yourself. The same thing with such links even in html emails.
If you use System.Uri.UnescapeDataString(yourString) it will ignore the +. This method should only be used in cases like yours where when the string was encoded using some sort of legacy approach either on the client or server.
See this blog post:
http://blogs.msdn.com/b/yangxind/archive/2006/11/09/don-t-use-net-system-uri-unescapedatastring-in-url-decoding.aspx
If you URLEncode the string before adding it to the URL you will not have any of those problems (the automatic URLDecode will return it to the original state).
Well, obviously you should have the Base64 string URLEncoded before sending it to the server.
If you cannot accomplish that, I would suggest simply replacing any embedded spaces back to +; since b64 strings are not suposed to have spaces, its a legitimate tactic...
System.Web.HttpUtility.UrlEncode(yourString) will do the trick.
As a quick hack you could replace space with plus character before base64-decoding.
I am by no means a C# developer but it looks like you need to url ENCODE your Base64 string before sending it as a url.
Can't you just assume a space is a + and replace it?
Request.QueryString["VLTrap"].Replace(" ", "+");
;)