We Keep Coding

Why does not .NET add parameter value to exception message, e.g. in int.Parse

Very often in .NET methods throw generic errors like e.g.
int.Parse("test")
throws an exception with this message:
Input string was not in a correct format.
Now it would save a lot of trouble for many people if it just had the parameter value to help debug things easier:
Input string "test" was not in a correct format.
This seems like a natural and easy thing to have, yet .NET does not do it in many places like e.g. parsing. Is there is any reason or conceptual problem with doing that or is it just a "missing feature"?

I suspect that the reason is primarily for security reasons. Some concerns with displaying/rendering the text to be parsed in the message returned are (but not limited to)
The text to be parsed may be very long. This is would be problematic from a memory usage and display perspective not to mention developer's habits of logging exception messages (not unreasonably).
The text may contain characters that mess with the formatting (e.g., tab, LF, CR, etc.)
The text may contain sensitive data. On this point, it's worth nothing that most developers, at least starting out, generally log or display error messages at an exception level by default. Not including the text here means there's no unintended data leak to catch the unwary.
It's conceivable (though unlikely) that an exploit could be found whereby a malformed piece of text could have a nasty unintended side effect.
Additionally, the value being parsed is being supplied by the caller which leaves them the option of deciding if it's best to log the content or not - it's not int.Parse()'s place to return the value in the exception message.
All in all, displaying a concise message without the originally supplied value is a judicious decision on part of MS to save us from ourselves as well as follow security best practices.

How to fix log forging in C#

I am using Log4net for logging into my web api application. I am using below code to log warnings and error in database.
public async Task<int> ExecuteNonQuery(string procedureName,params SqlParameter[] parameters)
{
try
{
this.logger.DebugFormat("{0} stating call",procedureName);
......
......
......
}
further code
When i ran fortify scan it gives me log forging issue. Is there any idea how we can resolve this issue. I tried below code but didn't work
procedureName = Regex.Replace(procedureName, "[^ A - Za - z0 - 9 $]", string.Empty);

Fortify is guarding against Log Forging/Log Injection, where a malicious user could get unescaped input into your system in such a way that it's written to your log file. The idea would be to add in mis-leading log entries that would throw automated or manual intrusion detection or intrusion investigation attempts off the scent of the malicious user and what they are really trying to accomplish.
In order to combat this, one must carefully control which portions of user input end up being used in a log file. How to do this varies by how strict you are. You could simply escape all user input in your log, stripping out things like newlines or control characters. To be extremely strict, one removes all user input from their logging statements and uses a set of pre-developed events that form a "whitelist" of events that can be logged in the system.
In your case, it sounds like Fortify isn't smart enough to know if procedureName comes from user input or not, so it flags your logging statement as potentially containing dynamic/user content. If you are positive that you know where procedureName comes from, then you can ignore this as a false positive. Otherwise, you can use one of the mitigation techniques I've listed to make your logging statement more secure.

The answer from Seafish might be a bit hard to dissect. For that I am going to try and answer this.
As Fortify will explain, the Log Forging issue is because Fortify believes you are using a value/property that came directly from a user. In such a case, the user, if they were a bad actor, might have set that value to confuse your logs which messes with your ability to perform a proper audit.
The resolutions are usually one or multiple of the following:
Stripping out newlines or control characters from all properties or variables coming from a user before it hits your logs.
Remove all User input from being put into the logs, and instead using predefined messages to output.
Along with the first method, you can wrap all user defined values within certain markers to identify a true log entry.
The last suggestion is more of something that Fortify will likely not understand, which is why I recommended doing so in combination with the first solution. Just like using double quotes to quote someone, you can use a series of brackets around values in logs to help make it harder to forge, as well as even encrypting the Date/time thats on each line and using that to start the line so you can confirm that the start of a line is where you have a properly decrypted date/time that matches the date/time in the output message.

C# Potentially Dangerous Error, Apostrophe during Form Submission

Please help me understand why this is happening. My Web Application retrieves text that contains Apostrophe from the database to display using a Label in a Web Form. However, it is getting a Potentially Dangerous Error when I submit the Web Form.
I can solve this error by using HttpUtility.HtmlDecode() on the value before I set it to Label.Text
I have been Googling, and I cannot understand the flow that causes this problem.
I understand that Potentially Dangerous Error occurs when submitting potential HTML tags like '', and it also filters some encoded characters like "'"
However, what I cannot understand are:
Does the string value automatically HTML encodes when it displays on the Web Form? Because the value from my database is not HTML encoded.
Why does decoding it solves the error when my value from the database is not encoded in the first place? If that is the case, should I decode all values from database?

I think you are looking for something like this -> How to allow apostrophe in asp.net website
The main reason this occurs is mostly due to fact that accepting apostrophes without handling it properly could lead to SQL Injection . By Default Microsoft enables a validation that occurs with every request, this occurs even before it reaches the normal page lifecycle, and checks if there is anything in the form that could potentially lead to SQL Injection or other OWASP vulnerabilities (In no way I am saying Microsoft protects you from everything, but at least tries to cover the basics and make you aware). By the end it is up to you to disable that functionality (based on my first link) but you should be aware and protect yourself in a proper manner.
On your questions enconding as the name says will take care of the apostrophe as a proper character not a special character that could still be used for a SQL query for instance. And obviously depending on the scenario you might want to decode or encode it again when you present it. If you want to display rich html then you probably need to decode, etc, etc.
Bottom line I think you should go back to my first paragraph and understand when this starts and only then you will be able to understand what would be the right solution for your case as I assume you dont want to risk creating a vulnerable website :)

Validating your site

What else needs to be validated apart from what I have below? This is my question.
It is important that any input to a site is properly validated:
Textboxes, etc – use .NET validators (or custom code if the validators aren’t appropriate)
Querystring or Form values – use manual validation (casting to specific types, boundary checking, etc)
This ties into the problems which XSS can reveal.
Basically you have to validate any input that someone could potentially tamper with:
Form Postbacks (mainly .NET Controls – these can be validated with .NET validation controls. Also if you have Request Validation turned on on all pages, this reduces the risk )
QueryString Values
Cookie values
HTTP Headers
Viewstate (automatically done for you as long as you have ViewState MAC enabled)
Javascript (all JS can be viewed and changed, so need to ensure no crucial functionality is handled by JavaScript- i.e. always enable server side validation)

There is a lot that can go wrong with a web application. Your list is pretty comprehensive, although it is duplication. The http spec only states, GET, POST, Cookie and Header. There are many different types of POST, but its all in the same part of the request.
For your list I would also add everything having to do with file upload, which is a type of POST. For instance, file name, mime type and the contents of the file. I would fire up a network monitoring application like Wireshark and everything in the request should be considered potentially harmful.
There will never be a one size fits all validation function. If you are merging sql injection and xss sanitation functions then you maybe in trouble. I recommend testing your site using automation. A free service like Sitewatch or an open source tool like skipfish will detect methods of attack that you have missed.
Also, on a side note. Passing the view state around with a MAC and/or encrypted is a gross misuse of cryptography. Cryptography is tool used when there is no other solution. By using a MAC or encryption you are opening the door for an attacker to brute force this value or use something like oracle padding attack to take advantage of you. A view state should be kept track by the server, period end of story.

I would suggest a different way of looking at the problem that is orthogonal to what you have here (and hence not incompatible, there's no reason why you can't examine it both ways in case you catch with one what you miss with another).
The two things that are important in any validation are:
Things you pay attention to.
Things you pass to another layer untouched.
Now, most of the things you've mentioned so far fit into the first cateogry. Cookies that you ignore fit into the second, as would query & post information if you passed to another handler with Server.Execute or similar.
The second category is the most debatable.
On the one hand, if a given handler (.aspx page, IHttpHandler, etc.) ignores a cookie that may be used by another handler at some point in the future, it's mostly up to that other handler to validate it.
On the other hand, it's always good to have an approach that assumes other layers have security holes and you shouldn't trust them to be correct, even if you wrote them yourself (especially if you wrote them yourself!)
A middle-ground position, is that if there are perhaps 5 different states some persistant data could validly be in, but only 3 make sense when a particular piece of code is hit, it might verify that it is in one of those 3 states, even if that doesn't pose a risk to that particular code.
That done, we'll concentrate on the first category.
Querystrings, form-data, post-backs, headers and cookies all fall under the same category of stuff that came from the user (whether they know it or not). Indeed, they are sometimes different ways of looking at the same thing.
Of this, there is a subset that we will actually work upon in any way.
Of that there is a range of legal values for each such item.
Of that, there is a range of legal combinations of values for the items as a whole.
Validation therefore becomes a matter of:
Identify what input we will act upon.
Make sure that each component of that input is valid in its own right.
Make sure that the combinations are valid (e.g it may be valid to not send a credit card number, but invalid to not send one but set payment type to "credit card").
Now, when we come to this, it's generally best not to try to catch certain attacks. For example, it's not so good to avoid ' in values that will be passed to SQL. Rather, we have three possibilities:
It's invalid to have ' in the value because it doesn't belong there (e.g. a value that can only be "true" or "false", or from a set list of values in which none of them contain '). Here we catch the fact that it isn't in the set of legal values, and ignore the precise nature of the attack (thus being protected also from other attacks we don't even know about!).
It's valid as human input, but not as what we will use. An example here is a large number (in some cultures ' is used to separate thousands). Here we canonicalise both "123,456,789" and "123'456'789" to 123456789 and don't care what it was like before that, as long as we can meaningfully do so (the input wasn't "fish" or a number that is out of the range of legal values for the case in hand).
It's valid input. If your application blocks apostrophes in name fields in an attempt to block SQL-injection, then it's buggy because there are real names with apostrophes out there. In this case we consider "d'Eath" and "O'Grady" to be valid input and deal with the fact that ' is significant in SQL by escaping properly (ideally by using an API for data access that will do this for us.
A classic example of the third point with ASP.NET is code that blocks "suspicious" input with < and > - something that makes a great number of ASP.NET pages buggy. Granted, it's better to be buggy in blocking that inappropriately than buggy by accepting it inappropriately, but the defaults are for people who haven't thought about validation and trying to stop them from hurting themselves too badly. Since you are thinking about validation, you should consider whether it's appropriate to turn that automatic validation off and then treat < and > in a manner appropriate for your given use.
Note also that I haven't said anything about javascript. I don't validate javascript (unless perhaps I was actually receiving it), I ignore it. I pretend it doesn't exist and then I won't miss a case where its validation could be tampered with. Pretend yours doesn't exist at this layer too. Ultimately client-side validation is to save the good guys making honest mistakes time, not to twart the bad guys.
For similar reasons, this is best not tested through a browser. Use Fiddler to construct requests that hit the validation points you want to examine. This way all client-side validation is by-passed, and you're looking at the server the same way an attacker will.
Finally, remember that a page with 100% perfect validation is not necessarily secure. E.g. if your validation is perfect but your authentication poor then someone can send "valid" code to it that will be just - perhaps more - nasty as the more classic SQL-injection of XSS code. That hits onto other topics that are for other questions, except that validation as discussed here is only part of the puzzle.

How can I return control to the client app and ask for input again from my API?

Writing a pure API seems to be bring up some challenges. For example, I am used to writing winforms/asp.net apps where if the input I have is invalid, I can programatically bring up a dialog box/webpage.
However, an API has no knowledge of the GUI app it may be executed from. If I have a method called TakeString (String s), and the string (s) cannot be more than 5 letters, how would I return control to the client if it was so? An exception seems like overkill? A simple return won't work if I am returning something in the method.
Thanks

An exception is exactly the right thing to do. You document that you'll only accept a string as input if it's 5 letters or fewer, and throw ArgumentException otherwise. Why would that be overkill?
In this case, you don't even need to worry about alternative mechanisms if the client can't detect the validity of what they're passing you: if the calling code has any doubt about the arguments they're going to pass you, it's not exactly beyond the wit of man for them to check it themselves.
Trying to make the API forgiving of invalid input is a recipe for disaster. Validate the input as stringently as you can, within reason. If you force the client to behave well, you'll have a lot fewer problems later on.

An Exception is the best way to handle this.
By definition, if the user is supplying invalid data, this is an exception. The client app needs to be responsible for handling the exception and prompting the user to re-enter as necessary.
Your API (as you already pointed out) should be ignorant of the client app, including what type of app it is, so all your API has to do is say "Hey, that's invalid!" and let the developer of the client app figure out how to handle it.
For a simple example if you try the following line of C# code:
int myInt = Convert.ToInt32("SSDS");
the .NET Framework doesn't try to re-prompt the user. It just throws an exception.