We Keep Coding

Replacing URL encoded data with their symbols

Hi I am trying to format my Url in order for it to look more user friendly.So far I managed to replace spaces with "-" but it seems that there are special characters like # and : that display as encoded data.This is what I mean:
http://localhost:51208/Home/Details/C%23-in-Depth%2c-Second-Edition/BookId-3
The "#" symbol is displayed as %23 and the "," is displayed as %2c.I would like to be able to replace this encoding with their original symbols.
Does such a way exist?

Oh no, you totally don't want to replace it with #. This symbol has a special meaning in an url. It represents the fragment identifier and its value is never sent to the server. This basically means that if there's a # symbol in your url, everything that follows it gets truncated and never sent to the server. You may take a look at the following post to see what StackOverflow uses to format the slug in the question title. You could run your string through this replace function in order to make sure that no dangerous characters are left.
I would also recommend you reading the following blog post from Scott Hanselman where he covers the various scenarios you might encounter with IIS if you attempt to send special characters in the path portion of your url. I am quoting his conclusion here:
After ALL this effort to get crazy stuff in the Request Path, it's
worth mentioning that simply keeping the values as a part of the Query
String (remember WAY back at the beginning of this post?) is easier,
cleaner, more flexible, and more secure.

Just replace "#" with "sharp" and ":" with "-", you cannot just put those special characters in the url

C# URL QueryString Trouble

I have a WP7 project where I am using the below code. It normally works ok, but I am getting a strange result with some particular strings being passed through.
Service = "3q%23L3t41tGfXQDTaZMbn%23w%3D%3D?f"
NavigationService.Navigate(new Uri("/Details.xaml?service=" + Service, UriKind.Relative));
Next Page:
NavigationContext.QueryString.TryGetValue("service", out Service1);
Service1 now = 3q#L3t41tGfXQDTaZMbn#w==?f
Why has the string changed?

The string hasn't changed, but you're looking at it in two different ways.
The way to encode 3q#L3t41tGfXQDTaZMbn#w==?f for as URI content is as 3q%23L3t41tGfXQDTaZMbn%23w%3D%3D?f. (Actually, it's 3q%23L3t41tGfXQDTaZMbn%23w%3D%3D%3Ff but you get away with the ? near the end not being properly escaped to %3F in this context).
Your means of writing the string, expects to receive it escaped.
Your means of reading the string, returns it unescaped.
Things are working pretty much perfectly, really.
When you need to write the string again, then just escape it again:
Service = Uri.EscapeDataString(Service1);

In your first code snippet the string is URL Encoded.
In the 2nd code snippet, the string is URL Decoded.
They are essentially the same strings, just with encoding applied/removed.
For example: urlencoding # you get %23
For further reading check out this wikipedia article on encoding.
Since HttpUtility isn't part of WP7 Silverlight stack, I'd recommend using Uri.EscapeUriString to escape any URI's that have not been escaped.

You should probably URL encode the string if you want it to pass through unscathed.

How to use strange characters in a query string

I am using silverlight / ASP .NET and C#. What if I want to do this from silverlight for instance,
// I have left out the quotes to show you literally what the characters
// are that I want to use
string password = vtakyoj#"5
string encodedPassword = HttpUtility.UrlEncode(encryptedPassword, Encoding.UTF8);
// encoded password now = vtakyoj%23%225
URI uri = new URI("http://www.url.com/page.aspx#password=vtakyoj%23%225");
HttpPage.Window.Navigate(uri);
If I debug and look at the value of uri it shows up as this (we are still inside the silverlight app),
http://www.url.com?password=vtakyoj%23"5
So the %22 has become a quote for some reason.
If I then debug inside the page.aspx code (which of course is ASP .NET) the value of Request["password"] is actually this,
vtakyoj#"5
Which is the original value. How does that work? I would have thought that I would have to go,
HttpUtility.UrlDecode(Request["password"], Encoding.UTF8)
To get the original value.
Hope this makes sense?
Thanks.

First lets start with the UTF8 business. Esentially in this case there isn't any. When a string contains characters with in the standard ASCII character range (as your password does) a UTF8 encoding of that string is identical to a single byte ASCII string.
You start with this:-
vtakyoj#"5
The HttpUtility.UrlEncode somewhat aggressively encodes it to:-
vtakyoj%23%225
Its encoded the # and " however only # has special meaning in a URL. Hence when you view string value of the Uri object in Silverlight you see:-
vtakyoj%23"5
Edit (answering supplementary questions)
How does it know to decode it?
All data in a url must be properly encoded thats part of its being valid Url. Hence the webserver can rightly assume that all data in the query string has been appropriately encoded.
What if I had a real string which had %23 in it?
The correct encoding for "%23" would be "%3723" where %37 is %
Is that a documented feature of Request["Password"] that it decodes it?
Well I dunno, you'd have check the documentation I guess. BTW use Request.QueryString["Password"] the presence of this same indexer directly on Request was for the convenience of porting classic ASP to .NET. It doesn't make any real difference but its better for clarity since its easier to make the distinction between QueryString values and Form values.
if I don't use UFT8 the characters are being filtered out.
Aare you sure that non-ASCII characters may be present in the password? Can you provide an example you current example does not need encoding with UTF-8?

If Request["password"] is to work, you need "http://url.com?password=" + HttpUtility.UrlEncode("abc%$^##"). I.e. you need ? to separate the hostname.
Also the # syntax is username:password#hostname, but it has been disabled in IE7 and above IIRC.

C# MVC: Trailing equal sign in URL doesn't hit route

I have an MVC route like this www.example.com/Find?Key= with the Key being a Base64 string. The problem is that the Base64 string sometimes has a trailing equal sign (=) such as:
huhsdfjbsdf2394=
When that happens, for some reason my route doesn't get hit anymore.
What should I do to resolve this?
My route:
routes.MapRoute(
"FindByKeyRoute",
"Find",
new { controller = "Search", action = "FindByKey" }
);
If I have http://www.example.com/Find?Key=bla then it works.
If I have http://www.example.com/Find?Key=bla= then it doesn't work anymore.
Important Addition:
I'm writing against an IIS7 instance that doesn't allow % or similar encoding. That's why I didn't use UrlEncode to begin with.

EDIT: Original suggestion which apparently doesn't work
I'm sure the reason is that it thinks it's a query parameter called Key. Could you make it a parameter, with that part being the value, e.g.
www.example.com/Find?Route=Key=
I expect that would work (as the parser would be looking for an & to start the next parameter) but it's possible it'll confuse things still.
Suggestion which I believe will work
Alternatively, replace "=" in the base64 encoded value with something else on the way out, and re-replace it on the way back in, if you see what I mean. Basically use a different base64 decodabet.
Alternative suggestion which should work
Before adding base64 to the URL:
private static readonly char[] Base64Padding = new char[] { '=' };
...
base64 = base64.TrimEnd(Base64Padding);
Then before calling Convert.FromBase64String() (which is what I assume you're doing) on the inbound request:
// Round up to a multiple of 4 characters.
int paddingLength = (4 - (base64.Length % 4)) % 4;
base64 = base64.PadRight(base64.Length + paddingLength, '=');

IF you're passing data in the URL you should probably URL Encode it which would take care of the trailing =.
http://www.albionresearch.com/misc/urlencode.php

UrlEncode the encrypted (it is encrypted, right?) parameter.
If it is an encrypted string, beware that spaces and the + character will also get in your way.
Ok, so IIS 7 won't allow some special characters as part of your path. However, it would allow them if they were part of the querystring.
It is apparently, possible, to change this with a reg hack, but I wouldn't recommend that.
What I would suggest, then, is to use an alternate token, as suggested by Mr Skeet, or simply do not use it in your path, use it as querystring, where you CAN url encode it.
If it is an encrypted string, you haven't verified that it is or is not, you may in some cases get other 'illegal' characters. Querystring really would be the way to go.
Except your sample shows it as querystring... So what gives? Where did you find an IIS that won't allow standard uri encoding as part of the querystring??
Ok then. Thanks for the update.
RequestFiltering?
I see. Still that mentions double-encoded values that it blocks. Someone created a URL Sequence to deny any request with the '%' characters? At that point you might want to not use the encrypted string at all, but generate a GUID or something else that is guaranteed to not contain special characters, yet is not trivial to guess.

Double/incomplete Parameter Url Encoding

In my web app, my parameters can contain all sorts of crazy characters (russian chars, slashes, spaces etc) and can therefor not always be represented as-is in a URL.
Sending them on their merry way will work in about 50% of the cases. Some things like spaces are already encoded somewhere (I'm guessing in the Html.BuildUrlFromExpression does). Other things though (like "/" and "*") are not.
Now I don't know what to do anymore because if I encode them myself, my encoding will get partially encoded again and end up wrong. If I don't encode them, some characters will not get through.
What I did is manually .replace() the characters I had problems with.
This is off course not a good idea.
Ideas?
--Edit--
I know there are a multitude of encoding/decoding libraries at my disposal.
It just looks like the mvc framework is already trying to do it for me, but not completely.
<a href="<%=Html.BuildUrlFromExpression<SearchController>(c=>c.Search("", 1, "a \v/&irdStr*ng"))%>" title="my hat's awesome!">
will render me
<a href="/Search.mvc/en/Search/1/a%20%5Cv/&irdStr*ng" title="my hat's awesome!">
Notice how the forward slash, asterisk and ampersand are not escaped.
Why are some escaped and others not? How can I now escape this properly?
Am I doing something wrong or is it the framework?

Parameters should be escaped using Uri.EscapeDataString:
string url = string.Format("http://www.foo.bar/page?name={0}&address={1}",
Uri.EscapeDataString("adlknad /?? lkm#"),
Uri.EscapeDataString(" qeio103 8182"));
Console.WriteLine(url);
Uri uri = new Uri(url);
string[] options = uri.Query.Split('?','&');
foreach (string option in options)
{
string[] parts = option.Split('=');
if (parts.Length == 2)
{
Console.WriteLine("{0} = {1}",parts[0],
Uri.UnescapeDataString(parts[1]));
}
}

AS others have mentioned, if you encode your string first you aviod the issue.
The MVC Framework is encoding characters that it knows it needs to encode, but leaving those that are valid URL characters (e.g. & % ? * /). This is because these are valid URL characters, although they are special chracters in a URL that might not acheive the result you are after.

Try using the Microsoft Anti-Cross Site Scripting library. It contains several Encode methods, which encode all the characters (including #, and characters in other languages). As for decoding, the browser should handle the encoded Url just fine, however if you need to manually decode the Url, use Uri.UnescapeDataString
Hope that helps.

Escaping of forward slahes and dots in path part of url is prohibited by security reason (althrough, it works in mono).

Html.BuildUrlFromExpression needs to be fixed then, would submit this upstream to the MVC project... alternatively do the encoding to the string before passing to BuildUrlFromExpression, and decode it when it comes back out on the other side.
It may not be readily fixable, as IIS may be handling the decoding of the url string beforehand... may need to do some more advanced encoding/decoding for alternative path characters in the utility methods, and decode on your behalf coming out.

I've seen similar posts on this. Too me, it looks like a flaw in MVC. The function would be more appropriately named "BuildUrlFromEncodedExpression". Whats worse, is that the called function needs to decode its input parameters. Yuk.
If there is any overlap between the characters encoded BuildUrlFromExpression() and the characters encoded by the caller (who, I think might fairly just encode any non-alphanumeric for simplicities sake) then you have potential for nasty bugs.

Server.URLEncode or HttpServerUtility.UrlEncode
I see what you're saying now - I didn't realize the question was specific to MVC. Looks like a limitation of that part of the MVC framework - particularly BuildUrlFromExpression is doing some URL encoding, but it knows that also needs some of those punctation as part of the framework URLs.
And also unfortunately, URLEncoding doesn't produce an invariant, i.e.
URLEncode(x) != URLEncode(URLEncode(x))
Wouldn't that be nice. Then you could pre-encode your variables and they wouldn't be double encoded.
There's probably an ASP.NET MVC framework best practice for this. I guess another thing you could do is encode into base64 or something that is URLEncode-invariant.

Have you tried using the Server.UrlEncode() method to do the encoding, and the Server.UrlDecode() method to decode?
I have not had any issues with using it for passing items.